Running continuous integration (CI) tests for microservice applications requires setting up real environment dependencies. If a microservice test suite requires access to a database (like PostgreSQL) or cache servers (like Redis), mocking these dependencies inside unit tests can mask query bugs. A comprehensive pipeline spins up real companion services in parallel containers using GitLab service definitions, executing integration tests against active dependencies.

By structuring GitLab CI stages with multi-container services, we can execute database integration tests reliably on every code push.

1. Structuring the GitLab CI Container Services

We write a .gitlab-ci.yml file configuring runners to launch PostgreSQL and Redis containers side-by-side with our test suite:

# .gitlab-ci.yml
stages:
  - test

run_integration_tests:
  stage: test
  image: python:3.9
  
  # Register service dependencies
  services:
    - name: postgres:13
      alias: db-host
    - name: redis:6-alpine
      alias: cache-host
      
  variables:
    POSTGRES_DB: "test_db"
    POSTGRES_USER: "test_user"
    POSTGRES_PASSWORD: "secret_password"
    DATABASE_URL: "postgresql://test_user:secret_password@db-host:5432/test_db"
    REDIS_URL: "redis://cache-host:6379/0"
    
  before_script:
    - apt-get update && apt-get install -y libpq-dev
    - pip install -r requirements.txt
    
  script:
    # Run tests using the environment variable connections
    - pytest tests/integration/

2. Caching Virtual Environments to Speed Up Runs

To reduce run durations, we define cache keys targeting local python packages (like pip and virtualenvs). This avoids re-downloading packages on subsequent runs, decreasing build queue execution times.