Building Production CI/CD Pipelines with GitLab CI/CD

The Goal: Automate Everything

A great CI/CD pipeline should: build fast, fail loudly, and deploy confidently. Here's how to build one with GitLab CI/CD.

Pipeline Structure

stages:
  - validate
  - test
  - build
  - scan
  - deploy

variables:
  DOCKER_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

Stage 1: Validate

Catch obvious errors before running expensive tests.

lint:
  stage: validate
  image: python:3.12-slim
  script:
    - pip install flake8 black
    - flake8 src/
    - black --check src/

Stage 2: Test

Run tests in parallel to keep pipelines fast.

unit-tests:
  stage: test
  image: python:3.12
  script:
    - pip install -r requirements-test.txt
    - pytest tests/unit/ --junitxml=report.xml
  artifacts:
    reports:
      junit: report.xml

integration-tests:
  stage: test
  services:
    - postgres:15
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: user
    POSTGRES_PASSWORD: secret
  script:
    - pytest tests/integration/

Stage 3: Build

Build and push Docker images to the registry.

build:
  stage: build
  image: docker:24
  services:
    - docker:dind
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $DOCKER_IMAGE .
    - docker push $DOCKER_IMAGE

Stage 4: Security Scanning

Never skip this in production pipelines.

trivy-scan:
  stage: scan
  image: aquasec/trivy:latest
  script:
    - trivy image --exit-code 1 --severity HIGH,CRITICAL $DOCKER_IMAGE

Stage 5: Deploy

Use environment-specific rules for controlled deployments.

deploy-staging:
  stage: deploy
  script:
    - kubectl set image deployment/web web=$DOCKER_IMAGE -n staging
  environment:
    name: staging
  only:
    - main

deploy-production:
  stage: deploy
  script:
    - kubectl set image deployment/web web=$DOCKER_IMAGE -n production
  environment:
    name: production
  when: manual
  only:
    - main

Key Best Practices

• Cache dependencies between runs to speed up pipelines
• Use needs: keyword to parallelize dependent stages
• Store sensitive values in GitLab CI/CD Variables (masked + protected)
• Set only: main for production deployments and require manual approval

With this structure, deployments become predictable, auditable, and safe.