This document provides detailed and professional CI/CD pipeline configurations for GitHub Actions, GitLab CI, and Jenkins. These configurations are designed to be comprehensive, covering essential stages such as linting, testing, building, and deployment, and are tailored for a typical modern application development workflow.
We will provide a generic structure and then specific examples for each platform, using a Node.js application with Docker as a common illustrative technology stack. These examples are designed to be highly adaptable to various other programming languages, frameworks, and deployment targets.
A robust CI/CD pipeline automates the software delivery process, ensuring quality, speed, and reliability. The common stages include:
Below, you will find detailed configurations for GitHub Actions, GitLab CI, and Jenkins. Each example demonstrates the core stages and includes best practices for secrets management, caching, and environment-specific deployments.
GitHub Actions provides a flexible and powerful way to automate workflows directly within your GitHub repository.
.github/workflows/main.yml**Key GitHub Actions Concepts & Best Practices:** * **`on`**: Defines when the workflow runs (push, pull request, manual `workflow_dispatch`). * **`jobs`**: Independent units of work that run in parallel by default. `needs` defines dependencies. * **`runs-on`**: Specifies the runner environment (e.g., `ubuntu-latest`). * **`uses`**: Reuses existing actions from the GitHub Marketplace (e.g., `actions/checkout`, `actions/setup-node`). * **`secrets`**: Securely store sensitive information (e.g., API keys, cloud credentials) accessible via `secrets.MY_SECRET`. `GITHUB_TOKEN` is a special token provided by GitHub. * **`environment`**: Links jobs to specific environments, enabling environment protection rules and providing a deployment URL. * **Caching**: Use `actions/cache` or built-in caching for package managers (like `npm`) to speed up builds. * **Conditional Logic**: `if` statements allow jobs or steps to run only under specific conditions (e.g., branch names, tag pushes). * **Artifacts**: `actions/upload-artifact` and `actions/download-artifact` to pass files between jobs or store build outputs. --- #### 2.2. GitLab CI Configuration GitLab CI/CD is tightly integrated with GitLab repositories, providing a powerful and flexible pipeline definition. * **File Location**: `.gitlab-ci.yml`
Workflow: DevOps Pipeline Generator
Step: gemini → analyze_infrastructure_needs
Description: This step identifies the foundational infrastructure requirements and critical decision points necessary to design and implement a robust, efficient, and secure CI/CD pipeline. Without specific project details, this analysis provides a comprehensive framework and outlines the crucial information needed to tailor a pipeline for your unique application.
This deliverable provides a detailed analysis of the infrastructure needs for generating a comprehensive CI/CD pipeline. It outlines the key domains of infrastructure that must be considered, from source code management and CI/CD platforms to build environments, deployment targets, and security. Given the generic request, this analysis focuses on establishing a robust framework for understanding and defining these needs. We highlight industry trends, provide strategic recommendations for optimal pipeline performance and security, and, most importantly, define the critical information required from your team to proceed with concrete pipeline generation. The subsequent steps will leverage this information to configure specific GitHub Actions, GitLab CI, or Jenkins pipelines.
The primary goal of this initial analysis is to systematically identify all infrastructure components and considerations that impact the design and functionality of a CI/CD pipeline. By understanding these needs upfront, we can ensure the generated pipeline is:
This step serves as a crucial bridge, translating the high-level goal of "DevOps Pipeline Generation" into actionable requirements for the subsequent configuration steps.
Building a resilient CI/CD pipeline requires careful consideration across several infrastructure domains. Each domain presents specific choices and implications for the final pipeline configuration.
The foundation of any CI/CD pipeline starts with where your code resides and the platform that orchestrates the pipeline.
* GitHub Actions:
* GitHub-hosted runners: Fully managed by GitHub, offering various OS (Ubuntu, Windows, macOS) and pre-installed software. Simpler setup, but less control and potentially higher cost for large-scale usage.
* Self-hosted runners: User-managed VMs or containers (Docker, Kubernetes) within your own infrastructure. Offers greater control over environment, security, and potentially lower cost for high-volume or specific hardware needs.
* GitLab CI:
* GitLab Shared Runners: Managed by GitLab, suitable for public projects and small private projects.
* GitLab Specific Runners: Self-hosted on your infrastructure (VMs, Docker, Kubernetes executor). Provides dedicated resources, custom environments, and enhanced security.
* Jenkins:
* Master-Agent Architecture: Requires a Jenkins master server and one or more agents (nodes) to execute jobs. Agents can be static VMs, Docker containers, or Kubernetes pods. High flexibility but requires significant management overhead.
The environment where your application is compiled, packaged, and tested is critical for consistency and reliability.
Once built, artifacts (e.g., Docker images, JARs, WARs, compiled binaries, npm packages) need to be stored securely and reliably.
The destination for your application, post-build and test, dictates many aspects of the deployment stage.
Security must be baked into the pipeline from the start, especially concerning sensitive credentials.
While primarily post-deployment, the pipeline should facilitate the integration of monitoring and logging tools to ensure visibility into application health and performance.
To generate a precise and effective CI/CD pipeline, we require specific details about your project and existing infrastructure. Please provide the following information:
* GitHub Actions
* GitLab CI
* Jenkins
* For GitHub Actions/GitLab CI:
* Managed/Cloud-hosted Runners (default, simpler)
* Self-hosted Runners (requires you to provide VMs/containers, more control)
* For Jenkins:
* Existing Jenkins Master/Agents
* New Jenkins setup (VMs, Docker, Kubernetes)
yaml
stages:
- lint
- test
- build
- deploy
variables:
NODE_VERSION: "18.x"
DOCKER_IMAGE_NAME: your-org/your-app # Replace with your Docker image name
REGISTRY: $CI_REGISTRY # GitLab's built-in registry, or specify an external one like docker.io
default:
image: node:${NODE_VERSION}-alpine # Default image for all jobs
cache:
key: ${CI_COMMIT_REF_SLUG} # Cache per branch/tag
paths:
- node_modules/
lint-job:
stage: lint
script:
- npm ci
- npm run lint
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: $CI_COMMIT_BRANCH =~ /^(develop|feature)\/.*$/
test-job:
stage: test
script:
- npm ci
- npm test
artifacts:
when: always
reports:
junit: test-results.xml # Path to JUnit XML test results
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: $CI_COMMIT_BRANCH =~ /^(develop|feature)\/.*$/
build-docker-image:
stage: build
image: docker:latest # Use a Docker image with Docker CLI installed
services:
- docker:dind # Docker-in-Docker service
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- >
if [ -n "$CI_COMMIT_TAG" ]; then
IMAGE_TAG="$CI_COMMIT_TAG"
else
IMAGE_TAG="$CI_COMMIT_SHORT_SHA"
fi
- docker build -t $REGISTRY/$DOCKER_IMAGE_NAME:$IMAGE_TAG -t $REGISTRY/$DOCKER_IMAGE_NAME:latest .
- docker push $REGISTRY/$DOCKER_IMAGE_NAME:$IMAGE_TAG
- docker push $REGISTRY/$DOCKER_IMAGE_NAME:latest
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Build on main branch
- if: $CI_COMMIT_BRANCH == "develop" # Build on develop branch
- if: $CI_COMMIT_TAG # Build on tags
deploy-dev:
stage: deploy
image: alpine/git:latest # A lightweight image with git for deployment scripts
environment
This document provides a detailed overview and illustrative examples of CI/CD pipeline configurations generated by the PantheraHive DevOps Pipeline Generator. Our aim is to equip you with robust, production-ready pipelines for GitHub Actions, GitLab CI, or Jenkins, encompassing essential stages like linting, testing, building, and deployment.
The DevOps Pipeline Generator is designed to streamline your CI/CD setup by automatically generating tailored configurations based on your project requirements. It ensures best practices are followed, providing a solid foundation for automated software delivery. This output details the structure, common stages, and how to implement these generated pipelines effectively.
Regardless of the chosen CI/CD platform, a modern DevOps pipeline typically incorporates several critical stages to ensure code quality, functionality, and reliable deployment. The generated pipelines include:
Below are conceptual outlines for GitLab CI and Jenkins, followed by a detailed example configuration for GitHub Actions. These examples demonstrate how the core stages are implemented across different platforms.
All generated pipelines follow a similar logical flow, adapted to the specific syntax and capabilities of the chosen platform:
main, pull request, scheduled).This example provides a comprehensive main.yml for a generic web application (e.g., Node.js, Python, or even a static site with build steps) using GitHub Actions. It includes linting, testing, building a Docker image, and deploying it to a container registry.
# .github/workflows/main.yml
name: CI/CD Pipeline for Web Application
on:
push:
branches:
- main
- develop
tags:
- 'v*' # Run on new tags like v1.0.0
pull_request:
branches:
- main
- develop
workflow_dispatch: # Allows manual trigger
env:
DOCKER_REGISTRY: ghcr.io # Or your Docker Hub/AWS ECR/GCR registry
IMAGE_NAME: ${{ github.repository }} # Uses repo name as image name
jobs:
lint_and_test:
name: Lint and Test
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js (Example for JS-based projects)
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm' # Or yarn, pnpm
cache-dependency-path: './package-lock.json' # Or yarn.lock, pnpm-lock.yaml
# You might have similar setup steps for Python, Java, etc.
# - name: Set up Python
# uses: actions/setup-python@v5
# with:
# python-version: '3.10'
# cache: 'pip'
# cache-dependency-path: './requirements.txt'
- name: Install dependencies
run: npm ci # Or pip install -r requirements.txt, mvn install
- name: Run Linter (e.g., ESLint, Flake8)
run: npm run lint # Or flake8 .
- name: Run Unit and Integration Tests
run: npm test # Or pytest
- name: Upload Test Results (e.g., for coverage reports)
uses: actions/upload-artifact@v4
if: always()
with:
name: test-results
path: |
./coverage/
./junit.xml # Example paths for common test output
build_docker_image:
name: Build Docker Image
runs-on: ubuntu-latest
needs: lint_and_test # Ensures linting and testing pass before building
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to Docker Registry
uses: docker/login-action@v3
with:
registry: ${{ env.DOCKER_REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} # Use GITHUB_TOKEN for GHCR, or specific secret for other registries
- name: Extract Docker metadata (tags, labels)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.DOCKER_REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha # Cache layers using GitHub Actions cache
cache-to: type=gha,mode=max
deploy_to_staging:
name: Deploy to Staging
runs-on: ubuntu-latest
needs: build_docker_image # Ensures image is built
environment: staging # Defines an environment for secrets/approvals
if: github.ref == format('refs/heads/{0}', github.event.repository.default_branch) # Deploy only from main branch
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download Artifacts (if needed, e.g., K8s manifests)
uses: actions/download-artifact@v4
with:
name: deployment-manifests
path: ./manifests
- name: Deploy to Staging (Example: Kubernetes using kubectl)
uses: azure/k8s-set-context@v3 # Or similar action for AWS/GCP
with:
method: kubeconfig
kubeconfig: ${{ secrets.KUBECONFIG_STAGING }}
# context: 'your-k8s-context'
- name: Apply Kubernetes manifests
run: |
kubectl apply -f ./manifests/deployment.yaml
kubectl apply -f ./manifests/service.yaml
kubectl rollout status deployment/your-app-staging
- name: Verify Staging Deployment
run: |
# Add commands to curl endpoint, check logs, etc.
echo "Staging deployment successful. Access at: http://your-staging-url.com"
# deploy_to_production:
# name: Deploy to Production
# runs-on: ubuntu-latest
# needs: deploy_to_staging # Or a manual approval step
# environment: production # Requires manual approval in GitHub Environments
# if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') # Trigger on tag push
# # You might need a manual approval step here via GitHub Environments
# steps:
# - name: Checkout repository
# uses: actions/checkout@v4
# - name: Deploy to Production (Example: using Helm)
# uses: azure/k8s-set-context@v3
# with:
# method: kubeconfig
# kubeconfig: ${{ secrets.KUBECONFIG_PRODUCTION }}
# - name: Upgrade Helm Chart
# run: |
# helm upgrade --install your-app ./helm-chart \
# --namespace production \
# --set image.tag=${{ github.ref_name }} \
# -f ./helm-chart/values-production.yaml
# - name: Verify Production Deployment
# run: |
# echo "Production deployment successful. Access at: http://your-production-url.com"
Key Features of this GitHub Actions Example:
lint_and_test, build_docker_image, and deploy_to_staging jobs.needs keyword ensures jobs run in correct order.env for reusable values.secrets.GITHUB_TOKEN and custom secrets (e.g., secrets.KUBECONFIG_STAGING).docker/metadata-action for intelligent tagging.if statements for deploying only from specific branches or tags.upload-artifact for test results, download-artifact for deployment manifests.workflow_dispatch allows for manual runs.For GitLab CI, the pipeline is defined in a .gitlab-ci.yml file at the root of your repository.
# .gitlab-ci.yml
image: docker:latest # Base image for jobs, can be changed per job
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
# Add other global variables here
stages:
- lint
- test
- build
- deploy_staging
- deploy_production
lint:
stage: lint
image: node:20 # Or python:3.10, etc.
script:
- npm ci
- npm run lint
# Only run on merge requests, or specific branches
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "develop"'
test:
stage: test
image: node:20
script:
- npm ci
- npm test
artifacts:
paths:
- coverage/
- junit.xml
expire_in: 1 week
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "develop"'
build_docker_image:
stage: build
image: docker:latest
services:
- docker:dind # Docker-in-Docker for building images
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $CI_REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHORT_SHA .
- docker push $CI_REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHORT_SHA
- docker tag $CI_REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_SHORT_SHA $CI_REGISTRY/$CI_PROJECT_PATH:latest # Tag latest
- docker push $CI_REGISTRY/$CI_PROJECT_PATH:latest
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
deploy_staging:
stage: deploy_staging
image: alpine/helm:3.10.0 # Or specific kubectl image
script:
- # Configure Kubernetes context using $KUBE_CONFIG secret variable
- echo "$KUBE_CONFIG_STAGING" | base64 -d > kubeconfig.yaml
- export KUBECONFIG=$(pwd)/kubeconfig.yaml
- helm upgrade --install my-app ./helm-chart -n staging --set image.tag=$CI_COMMIT_SHORT_SHA
environment:
name: staging
url: https://staging.example.com
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
deploy_production:
stage: deploy_production
image: alpine/helm:3.10.0