This document provides comprehensive CI/CD pipeline configurations for GitHub Actions, GitLab CI, and Jenkins, encompassing linting, testing, building, and deployment stages. This deliverable is designed to be directly actionable, offering detailed explanations and validation steps to ensure smooth integration into your development workflow.
This deliverable provides ready-to-use CI/CD pipeline configurations tailored for three popular platforms: GitHub Actions, GitLab CI, and Jenkins. Each configuration is designed to automate the software delivery process, ensuring code quality, reliability, and efficient deployment. The pipelines include essential stages:
For demonstration purposes, the examples provided assume a Node.js application, but the structure and principles are highly adaptable to other technology stacks (e.g., Python, Java, Go, .NET). Placeholders for specific credentials and deployment targets are clearly marked.
Before implementing these pipelines, please review the following:
npm install, npm run lint, npm run test, npm run build). You will need to adapt these commands to your specific language and framework (e.g., pip install, pytest, mvn clean install).eslint, jest). Adjust these to your project's tools.push events to specific branches (e.g., main, develop) or on pull_request/merge_request events. The examples focus on main branch triggers.GitHub Actions provides a powerful and flexible CI/CD solution directly within your GitHub repository. Workflows are defined in YAML files (.github/workflows/*.yml).
This GitHub Actions workflow will:
main branch and on pull requests..github/workflows/main.yml)
#### 3.1.3. Stage Breakdown
* **Setup**: Uses `actions/checkout@v4` to get the code and `actions/setup-node@v4` to set up the Node.js environment.
* **Install Dependencies**: `npm ci` ensures a clean installation of dependencies based on `package-lock.json`.
* **Linting**: `npm run lint` executes your configured linting script (e.g., ESLint).
* **Testing**: `npm run test` runs your test suite (e.g., Jest, Mocha).
* **Building**: `npm run build` compiles or bundles your application for production. An `upload-artifact` step is included to pass build artifacts between jobs.
* **Deployment**:
* `needs: build_and_test` ensures this job only runs if the previous job passed.
* `environment: production` links to a GitHub environment, allowing for protection rules (e.g., manual approvals).
* **AWS S3 Example**: Uses `aws-actions/configure-aws-credentials@v4` for authentication and `aws s3 sync` to upload static assets. CloudFront invalidation is included.
* **Docker/ECS Example (commented out)**: Demonstrates building and pushing a Docker image to ECR, then deploying to Amazon ECS.
#### 3.1.4. Validation Steps
1. **Repository Setup**: Ensure your project is hosted on GitHub.
2. **Add Workflow File**: Create the `.github/workflows/main.yml` file in your repository with the provided content.
3. **Define Secrets**:
* Go to your GitHub repository -> `Settings` -> `Secrets and variables` -> `Actions` -> `New repository secret`.
* Add `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` (or your chosen cloud provider credentials).
* If using GitHub Environments, configure secrets and protection rules under `Settings` -> `Environments`.
4. **Push to `main`**: Make a commit and push it to the `main` branch.
5. **Monitor Workflow**: Go to your repository -> `Actions` tab. You should see the "CI/CD Pipeline" workflow running.
6. **Verify Stages**: Check that each job (`Build and Test`, `Deploy to Production`) and its steps execute successfully.
7. **Verify Deployment**: Confirm that your application is updated in the target environment.
### 3.2. GitLab CI
GitLab CI/CD is tightly integrated with GitLab repositories, using a `.gitlab-ci.yml` file to define pipelines.
#### 3.2.1. Overview
This GitLab CI pipeline will:
1. Trigger on pushes to the `main` branch.
2. Define stages: `build`, `test`, `deploy`.
3. Set up Node.js.
4. Install dependencies, run linting, and execute tests within the `test` stage.
5. Build the application or Docker image within the `build` stage.
6. Deploy to a specified environment within the `deploy` stage.
#### 3.2.2. Configuration (`.gitlab-ci.yml`)
Workflow: DevOps Pipeline Generator
Step: gemini → analyze_infrastructure_needs
This document presents a comprehensive analysis of the foundational infrastructure requirements for establishing a robust and efficient CI/CD pipeline. As the initial phase of generating your DevOps pipeline, this analysis identifies critical components, general considerations, and industry best practices. It aims to provide a clear understanding of the architectural decisions necessary to support continuous integration, delivery, and deployment, ensuring scalability, security, and maintainability. While this step provides a general framework, subsequent steps will refine these needs based on your specific environment and preferences.
A well-designed CI/CD pipeline is only as effective as the underlying infrastructure that supports it. This analysis focuses on identifying the key infrastructure components and considerations essential for building, testing, packaging, and deploying software consistently and reliably. Understanding these needs upfront helps in selecting the right tools, optimizing costs, and ensuring the long-term success of your DevOps initiatives.
To generate an effective CI/CD pipeline, several core infrastructure components must be considered. Each plays a vital role in the end-to-end automation process.
The SCM system is the origin point for all CI/CD activities, triggering builds upon code changes.
* Reliable version control (Git-based preferred).
* Integration with chosen CI/CD platform (webhooks, APIs).
* Branching and merging capabilities.
* Access control and security features.
* Code review functionalities.
This is the central brain of the pipeline, defining and executing the workflow stages.
* Pipeline definition language (YAML, Groovy DSL, UI).
* Stage and step orchestration.
* Integration with SCM, build tools, testing frameworks, deployment targets.
* Scalable agent/runner management.
* Reporting and monitoring capabilities.
* Security for credentials and access.
These are the compute resources where actual build, test, linting, and packaging tasks are executed.
* Sufficient CPU, memory, and disk space for builds.
* Pre-installed tools (compilers, package managers, language runtimes, Docker).
* Scalability to handle concurrent builds.
* Isolation between builds (often achieved via containers or ephemeral VMs).
* Network access to SCM, artifact repositories, and deployment targets.
* Managed Runners: Provided by cloud CI/CD platforms (e.g., GitHub-hosted runners, GitLab Shared Runners).
* Self-hosted Runners/Agents: VMs (AWS EC2, Azure VMs, GCP Compute Engine), Kubernetes clusters (EKS, AKS, GKE) running agents.
* Containerization: Using Docker containers for isolated and reproducible build environments.
Repositories for storing build outputs (JARs, WARs, NuGet packages, Docker images, etc.).
* High availability and durability.
* Scalable storage capacity.
* Access control and versioning.
* Integration with CI/CD platform and deployment tools.
* Vulnerability scanning for container images.
* Container Registries: Docker Hub, Amazon ECR, Azure Container Registry, Google Container Registry, GitLab Container Registry.
* Artifact Repositories: JFrog Artifactory, Sonatype Nexus, AWS CodeArtifact.
The environments where the application will be deployed (development, staging, production).
* Appropriate compute resources (VMs, containers, serverless functions).
* Network configuration (load balancers, firewalls, DNS).
* Security considerations (IAM, network segmentation).
* Configuration management (Ansible, Chef, Puppet, Terraform).
* Virtual Machines: AWS EC2, Azure VMs, GCP Compute Engine.
* Container Orchestration: Kubernetes (EKS, AKS, GKE, OpenShift).
* Serverless Platforms: AWS Lambda, Azure Functions, Google Cloud Functions.
* Platform as a Service (PaaS): AWS Elastic Beanstalk, Azure App Service, Google App Engine, Heroku.
Securely storing and injecting sensitive information (API keys, database credentials, tokens) into the pipeline.
* Strong encryption at rest and in transit.
* Granular access control (least privilege).
* Auditing capabilities.
* Integration with CI/CD platform.
Observability solutions to track pipeline execution, application performance, and infrastructure health.
* Centralized log aggregation.
* Real-time metrics and dashboards.
* Alerting capabilities.
* Traceability across pipeline stages and deployed applications.
Integrating security scans and compliance checks throughout the pipeline ("Shift Left").
* Static Application Security Testing (SAST).
* Dynamic Application Security Testing (DAST).
* Software Composition Analysis (SCA) for open-source vulnerabilities.
* Container image scanning.
* Infrastructure as Code (IaC) security scanning.
* Compliance reporting.
Beyond individual components, several overarching factors influence infrastructure choices.
Based on typical industry trends and best practices, we recommend the following foundational approaches:
Insight:* Organizations increasingly prefer integrated, scalable, and low-maintenance CI/CD platforms.
Insight:* Container-based builds offer reproducibility and isolation, reducing "it works on my machine" issues. Kubernetes is a primary deployment target for new applications.
Insight:* Proactive security identification reduces remediation costs and risks significantly.
Insight:* Better observability leads to faster troubleshooting and improved system reliability.
Insight:* IaC is fundamental for achieving true CI/CD for infrastructure, enabling faster provisioning and disaster recovery.
To proceed with generating your tailored CI/CD pipeline, we require more specific information about your existing environment and preferences. Please provide details on the following:
Your input on these points will enable us to move to Step 2: Define CI/CD Tool and Workflow and generate a
This document provides comprehensive, detailed CI/CD pipeline configurations for GitHub Actions, GitLab CI, and Jenkins. These configurations are designed to cover standard DevOps practices including linting, testing, building, and deployment, and are presented with clear explanations and guidance for adaptation.
For consistency and clarity, all examples assume a Node.js application that is built into a Docker image and deployed to generic Staging and Production environments. You can adapt these examples to fit your specific technology stack and deployment targets.
A robust CI/CD pipeline is fundamental for modern software development, enabling automated and efficient delivery of applications. This output delivers actionable pipeline configurations for three leading platforms:
Each configuration includes stages for:
GitHub Actions allows you to automate, customize, and execute your software development workflows directly in your repository. Workflows are defined using YAML files and run on events like pushes or pull requests.
.yml file in .github/workflows/.push, pull_request, workflow_dispatch)..github/workflows/main.yml)Create this file in your repository: .github/workflows/main.yml
name: Node.js CI/CD Pipeline
on:
push:
branches:
- main
- develop
pull_request:
branches:
- main
- develop
workflow_dispatch: # Allows manual triggering of the workflow
env:
NODE_VERSION: '16.x' # Specify Node.js version
DOCKER_IMAGE_NAME: my-node-app # Name for your Docker image
jobs:
lint:
name: Lint Code
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm' # Cache npm dependencies
- name: Install Dependencies
run: npm ci
- name: Run Lint
run: npm run lint
test:
name: Run Tests
runs-on: ubuntu-latest
needs: lint # This job depends on the 'lint' job completing successfully
steps:
- name: Checkout Repository
uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
- name: Install Dependencies
run: npm ci
- name: Run Unit and Integration Tests
run: npm test
build_and_push_docker:
name: Build & Push Docker Image
runs-on: ubuntu-latest
needs: test # This job depends on the 'test' job completing successfully
if: github.event_name == 'push' # Only build and push on push events
steps:
- name: Checkout Repository
uses: actions/checkout@v3
- name: Log in to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_TOKEN }}
- name: Build and Push Docker Image
uses: docker/build-push-action@v4
with:
context: .
push: true
tags: ${{ secrets.DOCKER_USERNAME }}/${{ env.DOCKER_IMAGE_NAME }}:${{ github.sha }}
# Also tag with 'latest' for main branch pushes
tags: |
${{ secrets.DOCKER_USERNAME }}/${{ env.DOCKER_IMAGE_NAME }}:${{ github.sha }}
${{ github.ref == 'refs/heads/main' && format('{0}/{1}:latest', secrets.DOCKER_USERNAME, env.DOCKER_IMAGE_NAME) || '' }}
deploy_staging:
name: Deploy to Staging
runs-on: ubuntu-latest
needs: build_and_push_docker
if: github.event_name == 'push' # Only deploy on push events
environment:
name: Staging # Links to a GitHub Environment for protection rules
url: https://staging.your-app.com # Optional: URL for the deployed environment
steps:
- name: Checkout Repository
uses: actions/checkout@v3
- name: Configure AWS Credentials (Example for AWS ECR/ECS/EKS)
uses: aws-actions/configure-aws-credentials@v2
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Deploy to Staging Environment
yaml
image: node:18-alpine # Use a Node.js Docker image as base for all jobs
stages:
- build
- test
- deploy
variables:
# Define global variables
NPM_CACHE_DIR: "$CI_PROJECT_DIR/.npm"
# Optional: DOCKER_IMAGE_NAME: "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA"
cache:
key: ${CI_COMMIT_REF_SLUG} # Cache per branch/MR
paths:
- node_modules/
- .npm/ # Cache npm packages globally
before_script:
- echo "Starting CI/CD for $CI_PROJECT_NAME on branch $CI_COMMIT_REF_NAME"
- npm config set cache $NPM_CACHE_DIR
- npm ci --cache $NPM_CACHE_DIR # Install dependencies, using cache
- echo "Dependencies installed."
build_job:
stage: build
script:
- echo "Running build stage..."
- npm run build
- echo "Build completed."
artifacts:
paths:
- build/ # Adjust path to your build output directory
expire_in: 1 day # How long to keep the artifact
only:
- main
lint_job:
stage: test
script:
- echo "Running lint stage..."
- npm run lint
- echo "Lint completed."
only:
- main
- merge_requests
test_job:
stage: test
script:
- echo "Running test stage..."
- npm run test
- echo "Tests completed."
coverage: '/All files[^|]\|[^|]\s+([\d\.]+)/' # Example regex for coverage parsing
only:
- main
- merge_requests
deploy_staging_job:
stage: deploy
image: alpine/git # Use a minimal image for deployment (if no Node needed)
script:
- echo "Deploying to staging environment..."
- apt-get update && apt-get install -y rsync openssh-client # Install deployment tools
- eval "$(ssh-agent -s)"
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh