This deliverable provides detailed, production-ready CI/CD pipeline configurations for three leading platforms: GitHub Actions, GitLab CI, and Jenkins. These configurations are designed to automate the entire software delivery lifecycle, encompassing linting, testing, building, and deployment stages for a typical application (e.g., a containerized web service).
The following assumptions have been made to generate these configurations. Please review and adjust the provided code snippets to match your specific project requirements:
npm for Node.js examples (npm install, npm run lint, npm test, npm run build), but the principles apply to Maven, Gradle, pip, etc.lint script is available (e.g., npm run lint for ESLint).test script is available (e.g., npm test for Jest/Mocha).GitHub Actions provides a flexible and powerful way to automate workflows directly within your GitHub repository.
This GitHub Actions workflow will:
main and pull requests.main.yml ConfigurationCreate this file as .github/workflows/main.yml in your repository.
#### 2.3. Explanation of Stages
* **Lint and Test (`lint-test`)**:
* Checks out the code.
* Sets up Node.js (adjust version as needed).
* Caches `node_modules` to speed up subsequent runs.
* Installs dependencies using `npm ci` (clean install).
* Executes linting (`npm run lint`).
* Executes unit tests (`npm test`).
* **Build and Push Docker Image (`build-and-push-docker`)**:
* Depends on `lint-test` passing.
* Only runs on `main` branch pushes.
* Logs into Docker Hub using secrets (`DOCKER_USERNAME`, `DOCKER_PASSWORD`).
* Uses `docker/metadata-action` to generate proper Docker image tags (e.g., `latest` for `main`, and a SHA-based tag).
* Builds and pushes the Docker image to the specified registry. Utilizes GitHub Actions caching for Docker layers.
* **Deploy to Environment (`deploy`)**:
* Depends on `build-and-push-docker` passing.
* Only runs on `main` branch pushes.
* Uses a GitHub Environment (`production`) for protection rules (e.g., manual approval).
* **Placeholder for actual deployment logic**: This step requires customization based on your deployment target (Kubernetes, AWS, Azure, GCP, on-premise servers, etc.). It typically involves using `kubectl`, AWS CLI, Azure CLI, `ssh`, or specific deployment tools.
---
### 3. GitLab CI Configuration
GitLab CI is deeply integrated into GitLab, offering powerful and flexible CI/CD capabilities.
#### 3.1. Overview
This GitLab CI pipeline will:
* Trigger on pushes to `main` and merge requests.
* Define stages for linting, testing, building, and deploying.
* Build and push a Docker image to the GitLab Container Registry.
* Deploy the application to a target environment.
#### 3.2. `.gitlab-ci.yml` Configuration
Create this file as `.gitlab-ci.yml` in the root of your repository.
Workflow: DevOps Pipeline Generator
Step: gemini → analyze_infrastructure_needs
Description: This analysis identifies the core infrastructure components and capabilities required to support robust, scalable, and secure CI/CD pipelines, laying the groundwork for generating specific configurations.
To effectively generate complete CI/CD pipeline configurations for GitHub Actions, GitLab CI, or Jenkins, a thorough understanding of underlying infrastructure needs is paramount. This initial analysis focuses on identifying the essential infrastructure categories and capabilities that any modern DevOps pipeline requires to support stages like testing, linting, building, and deployment. Our goal is to ensure the generated pipelines are not only functional but also secure, scalable, and maintainable.
A comprehensive CI/CD pipeline relies on a diverse set of infrastructure components. Below are the critical categories and their associated requirements:
* GitHub Actions: Cloud-native, YAML-based, tightly integrated with GitHub. Leverages GitHub-hosted or self-hosted runners.
* GitLab CI: Built into GitLab, YAML-based, supports shared or specific runners. Offers comprehensive features from code to deployment.
* Jenkins: Highly extensible, open-source, Java-based. Requires dedicated infrastructure for the Jenkins controller and agents. Offers vast plugin ecosystem.
* Cloud-Hosted/Managed Runners: Provided by GitHub, GitLab, often sufficient for common workloads, no infrastructure to manage.
* Self-Hosted Runners/Agents: Required for specific hardware, network access, or custom toolchains. Can be VMs, Kubernetes pods, or physical servers.
* Scalability: Auto-scaling groups or Kubernetes Horizontal Pod Autoscalers (HPAs) for dynamic capacity.
* Ephemerality: Disposable environments per job for security and consistency.
* Resource Allocation: Adequate CPU, RAM, and disk I/O.
* Containerization: Using Docker images as build environments ensures consistency and isolation.
* Generic Artifact Repositories: Nexus Repository Manager, JFrog Artifactory.
* Container Registries: Docker Hub, Amazon ECR, Azure Container Registry, Google Container Registry/Artifact Registry, GitLab Container Registry.
* Language-Specific: Maven Central, npm registry, PyPI.
* Virtual Machines (VMs): AWS EC2, Azure VMs, Google Compute Engine.
* Container Orchestration: Kubernetes (AWS EKS, Azure AKS, Google GKE, OpenShift).
* Serverless Platforms: AWS Lambda, Azure Functions, Google Cloud Functions.
* Platform as a Service (PaaS): AWS Elastic Beanstalk, Azure App Service, Google App Engine, Heroku.
* Cloud-Native: AWS Secrets Manager, Azure Key Vault, Google Secret Manager.
* Dedicated Solutions: HashiCorp Vault.
* CI/CD Platform Built-in: GitHub Actions Secrets, GitLab CI/CD Variables (masked/protected), Jenkins Credentials.
* Pipeline Logs: Accessible directly from the CI/CD platform.
* Application Logs: Centralized logging (ELK stack, Splunk, Datadog, CloudWatch Logs, Azure Monitor Logs, Google Cloud Logging).
* Metrics & Dashboards: Prometheus/Grafana, Datadog, New Relic, cloud-native monitoring services.
* Alerting: PagerDuty, Slack, email integrations.
* SAST: SonarQube, Checkmarx, Fortify.
* SCA: Snyk, Trivy, Dependabot, Renovate.
* Linting: ESLint, Black, Flake8, RuboCop.
Based on this analysis, we recommend the following strategic approaches for your DevOps pipelines:
To generate the most accurate and effective CI/CD pipeline configurations, we require further details regarding your specific project context. Please provide the following information:
* e.g., Java Spring Boot, Node.js Express, Python Django/Flask, .NET Core, Go, React/Angular/Vue frontend, Mobile (iOS/Android).
* Specific build tools (e.g., Maven, Gradle, npm, yarn, pip, dotnet CLI).
* GitHub Actions
* GitLab CI
* Jenkins
* Cloud Provider: AWS, Azure, GCP, On-Premises.
* Deployment Model: Kubernetes (EKS, AKS, GKE), Serverless (Lambda, Azure Functions), VMs, PaaS (App Service, Elastic Beanstalk), S3/Blob Storage for static sites.
* Do you already use an artifact repository (e.g., Artifactory, Nexus)?
* Do you have a preferred secret management solution?
* Are there existing code quality or security scanning tools in use?
* Any compliance regulations (e.g., GDPR, HIPAA, SOC2).
* Performance or scalability targets.
* Specific testing frameworks or methodologies.
Once this information is provided, we can proceed to Step 2: "define_pipeline_stages" and then Step 3: "generate_pipeline_configurations".
lint, test, build, deploy. * DOCKER_IMAGE_NAME: Automatically set to GitLab's built-in registry image path.
* DOCKER_TAG: Uses the short commit SHA for unique image tagging.
* image: Specifies the base Docker image for jobs (e.g., node:20-alpine).
* before_script: Commands to run before each job, here used for npm ci and caching.
node_modules and .npm to speed up builds.lint_job): * Runs npm run lint.
* rules: Executes on pushes to main or on merge requests.
test_job): * Runs npm test.
* rules: Executes on pushes to main or on merge requests.
build_docker_image): * Uses docker:latest image with docker:dind service for Docker-in-Docker functionality.
* Logs into GitLab Container Registry using predefined CI/CD variables.
* Builds the Docker image with a SHA tag and latest tag (for main branch).
* Pushes both tagged images to the registry.
* rules: Only runs on pushes to main.
deploy_production): * Uses a minimal alpine/git image for potential deployment scripts.
* Placeholder for actual deployment logic: Similar to GitHub Actions, this section needs customization. Examples for Kubernetes (kubectl) and SSH deployment are commented out.
* environment: Defines a GitLab environment, providing links and history.
* when: manual: Sets production deployment to require manual approval for safety.
Jenkins is a highly extensible automation server, widely used for CI/CD. This configuration uses a Declarative Pipeline.
This Jenkins Pipeline (Jenkinsfile) will:
This document provides a detailed, professional, and actionable Continuous Integration/Continuous Delivery (CI/CD) pipeline configuration. We have generated a robust pipeline incorporating best practices for linting, testing, building, and deployment, designed to enhance your development workflow, ensure code quality, and accelerate releases.
The primary example provided will be for GitHub Actions, given its widespread adoption and tight integration with GitHub repositories. We will also include guidance on adapting this configuration for GitLab CI and Jenkins.
The generated pipeline automates key stages of your software delivery lifecycle, from code commit to production deployment. Its core objectives are:
Your CI/CD pipeline is structured into distinct, logical stages, each with specific responsibilities:
Lint:* Purpose: Analyze code for programmatic errors, bugs, stylistic errors, and suspicious constructs.
* Benefits: Enforces coding standards, improves readability, and catches potential issues early.
Test:* Purpose: Execute unit, integration, and (optionally) end-to-end tests to verify application functionality.
* Benefits: Ensures code changes don't introduce regressions and that new features work as expected.
Build:* Purpose: Compile source code, resolve dependencies, create deployable artifacts (e.g., JAR, WAR, executable, front-end bundles), and build Docker images.
* Benefits: Creates a consistent, versioned artifact ready for deployment.
Deploy:* Purpose: Distribute the built artifacts to target environments (e.g., Development, Staging, Production). This often involves updating cloud services, Kubernetes clusters, or virtual machines.
* Benefits: Automates releases, reduces manual errors, and ensures consistent deployments.
This section provides a complete GitHub Actions workflow (.github/workflows/main.yml) for a typical Node.js web application that uses Docker for containerization and deploys to AWS Elastic Container Service (ECS).
Before implementing this pipeline, ensure the following are configured:
* Purpose: Securely authenticate GitHub Actions with AWS without storing long-lived AWS credentials in GitHub Secrets.
* Setup: Create an IAM OIDC provider for your GitHub repository. Then, create an IAM role with a trust policy that allows token.actions.githubusercontent.com to assume the role, conditioned on your repository and environment. Attach policies to this role that grant permissions for ECR (push image) and ECS (update service).
* Reference: [AWS Documentation for OIDC](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services)
* AWS_REGION: Your AWS region (e.g., us-east-1).
* AWS_ACCOUNT_ID: Your AWS account ID.
* ECR_REPOSITORY_NAME: The name of your Elastic Container Registry repository.
* ECS_CLUSTER_NAME: The name of your ECS cluster.
* ECS_SERVICE_NAME: The name of your ECS service.
* ECS_TASK_DEFINITION_FAMILY: The family name of your ECS task definition.
* (Optional) DOCKERHUB_USERNAME, DOCKERHUB_TOKEN: If pushing to Docker Hub instead of ECR.
.github/workflows/main.yml)Create a file named main.yml (or any other descriptive name) inside the .github/workflows/ directory of your repository.
name: CI/CD Pipeline
on:
push:
branches:
- main # Trigger on pushes to the main branch
pull_request:
branches:
- main # Trigger on pull requests targeting the main branch
workflow_dispatch: # Allows manual triggering of the workflow
env:
NODE_VERSION: '18.x' # Specify Node.js version
DOCKER_IMAGE_NAME: ${{ secrets.ECR_REPOSITORY_NAME }} # ECR repository name from secrets
AWS_REGION: ${{ secrets.AWS_REGION }} # AWS region from secrets
AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }} # AWS account ID from secrets
jobs:
lint:
name: Lint Code
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm' # Cache npm dependencies
- name: Install dependencies
run: npm ci # Use npm ci for clean installs in CI
- name: Run ESLint
run: npm run lint # Assuming you have a 'lint' script in package.json
# Example: "lint": "eslint . --ext .js,.jsx,.ts,.tsx"
test:
name: Run Unit and Integration Tests
runs-on: ubuntu-latest
needs: lint # This job depends on the 'lint' job completing successfully
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm test # Assuming you have a 'test' script in package.json
# Example: "test": "jest"
build_and_push_docker:
name: Build & Push Docker Image
runs-on: ubuntu-latest
needs: test # This job depends on the 'test' job completing successfully
permissions:
id-token: write # Required for OIDC authentication with AWS
contents: read # Required to checkout code
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ env.AWS_ACCOUNT_ID }}:role/GitHubActionsOIDC-Role # Replace with your IAM Role ARN
aws-region: ${{ env.AWS_REGION }}
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
- name: Build Docker image
id: build-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
IMAGE_TAG: ${{ github.sha }} # Use commit SHA as image tag
run: |
docker build -t $ECR_REGISTRY/$DOCKER_IMAGE_NAME:$IMAGE_TAG .
docker tag $ECR_REGISTRY/$DOCKER_IMAGE_NAME:$IMAGE_TAG $ECR_REGISTRY/$DOCKER_IMAGE_NAME:latest # Also tag as latest
- name: Push Docker image to ECR
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
IMAGE_TAG: ${{ github.sha }}
run: |
docker push $ECR_REGISTRY/$DOCKER_IMAGE_NAME:$IMAGE_TAG
docker push $ECR_REGISTRY/$DOCKER_IMAGE_NAME:latest
- name: Generate image tag output
id: set-image-tag
run: echo "image_tag=${{ github.sha }}" >> $GITHUB_OUTPUT
deploy_to_ecs:
name: Deploy to AWS ECS
runs-on: ubuntu-latest
needs: build_and_push_docker # This job depends on the Docker image being pushed
environment: production # Designate this as a production deployment, can require manual approval
permissions:
id-token: write # Required for OIDC authentication with AWS
contents: read # Required to checkout code
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ env.AWS_ACCOUNT_ID }}:role/GitHubActionsOIDC-Role # Replace with your IAM Role ARN
aws-region: ${{ env.AWS_REGION }}
- name: Download task definition
id: download-task-definition
run: |
aws ecs describe-task-definition --task-definition ${{ secrets.ECS_TASK_DEFINITION_FAMILY }} \
--query taskDefinition > task-definition.json
- name: Fill in the new image ID in the Amazon ECS task definition
id: render-task-definition
uses: aws-actions/amazon-ecs-render-task-definition@v1
with:
task-definition: task-definition.json
container-name: your-app-container-name # Replace with the name of your container in the task definition
image: ${{ steps.login-ecr.outputs.registry }}/${{ env.DOCKER_IMAGE_NAME }}:${{ needs.build_and_push_docker.outputs.image_tag }}
- name: Deploy Amazon ECS task definition
uses: aws-actions/amazon-ecs-deploy-task-definition@v1
with:
task-definition: ${{ steps.render-task-definition.outputs.task-definition }}
service: ${{ secrets.ECS_SERVICE_NAME }}
cluster: ${{ secrets.ECS_CLUSTER_NAME }}
wait-for-service-stability: true # Wait for the ECS service to become stable
* Python: Replace actions/setup-node@v4 with actions/setup-python@v5. Use pip install -r requirements.txt instead of npm ci, and pytest or flake8 for linting/testing commands.
* Java: Use actions/setup-java@v4. Use mvn clean install or gradle build.
* Go: Use actions/setup-go@v5. Use go test ./... and go build.
npm run lint and npm test to match your project's specific commands (e.g., yarn lint, python -m pytest, mvn test). * Google Cloud Run/GKE: Use google-github-actions/auth@v2 for OIDC, google-github-actions/setup-gcloud@v2, and gcloud run deploy or kubectl apply.
* Azure App Service/AKS: Use azure/login@v1 for OIDC, and azure/webapps-deploy@v2 or kubectl apply.
* Serverless Framework: Add a step to npm install -g serverless and then serverless deploy.
staging or development environments, you can duplicate the deploy_to_ecs job, change the environment: to staging or development, and adjust the target ECS cluster/service names accordingly. You can also add conditional deployments based on branches (e.g., main deploys to production, develop deploys to staging)..gitlab-ci.yml)GitLab CI uses a .gitlab-ci.yml file in the root of your repository. It leverages stages and jobs.
stages:
- lint
- test
- build
- deploy:development
- deploy:production
variables:
DOCKER_IMAGE_NAME: $CI_REGISTRY_IMAGE
AWS_REGION: us