Docker Compose: Orchestrate Multi-Container Apps Like a Pro

From Docker Chaos to Compose Clarity

The Orchestra Conductor Analogy

Think of it like this: An orchestra has violins, cellos, drums, and flutes. Each musician plays individually, but the conductor ensures they start together, play in harmony, and follow the same tempo. Docker Compose is your conductor—defining how all your containers work together, starting and stopping them as one cohesive application.

What is Docker Compose?

• Declarative: Define what you want, not how to do it
• Version controlled: Your infrastructure lives in code
• Reproducible: Anyone can run your stack with one command
• Environment consistency: Development matches production setup

Anatomy of a Compose File

1. Services

2. Networks

3. Volumes

yaml
version: '3.8'

services:
  # Service definitions go here

networks:
  # Network definitions go here

volumes:
  # Volume definitions go here

Anatomy of a Docker Compose File: Complete Line by Line Explanation

yaml
version: '3.8'

services:
  web:
    build:
      context: ./frontend
      dockerfile: Dockerfile.prod
      args:
        NODE_VERSION: 18
    image: myapp-web:latest
    container_name: web_app
    ports:
      - "3000:3000"
      - "3001:3001"
    environment:
      NODE_ENV: production
      API_URL: http://api:8080
    env_file:
      - .env.production
    volumes:
      - ./frontend/src:/app/src
      - node_modules:/app/node_modules
    depends_on:
      - api
      - redis
    networks:
      - frontend
      - backend
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  api:
    image: node:18-alpine
    working_dir: /app
    command: npm start
    volumes:
      - ./api:/app
    environment:
      DB_HOST: database
      REDIS_URL: redis://redis:6379
    depends_on:
      database:
        condition: service_healthy
    networks:
      - backend
    restart: always

  database:
    image: postgres:15-alpine
    container_name: postgres_db
    volumes:
      - db_data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: myapp
    ports:
      - "5432:5432"
    networks:
      - backend
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myuser"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data
    networks:
      - backend

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true

volumes:
  db_data:
    driver: local
  redis_data:
  node_modules:

1. Top-Level Elements

2. The web Service (Frontend)

3. The api & database Services

4. The redis Service

5. Networks & Volumes (Top-Level Definitions)

Building from Basic to Advanced

Level 1: Simple Web Application

yaml
version: '3.8'

services:
  web:
    # Why: Use pre-built image from Docker Hub
    image: node:18-alpine
    # Why: Map host port 3000 to container port 3000
    ports:
      - "3000:3000"
    # Why: Mount current directory for live code updates
    volumes:
      - ./app:/app
    # Why: Set working directory inside container
    working_dir: /app
    # Why: Start the application
    command: npm start
    # Why: Set environment variables
    environment:
      - NODE_ENV=development
      - DB_HOST=database
    # Why: Ensure database starts before web app
    depends_on:
      - database

  database:
    # Why: Use PostgreSQL 15
    image: postgres:15-alpine
    # Why: Persist database data
    volumes:
      - db_data:/var/lib/postgresql/data
    # Why: Configure database credentials
    environment:
      - POSTGRES_USER=myuser
      - POSTGRES_PASSWORD=mypassword
      - POSTGRES_DB=myapp

volumes:
  # Why: Named volume for database persistence
  db_data:

1. Compose creates a default network
2. Starts database service first (due to depends_on)
3. Starts web service
4. Both containers can communicate via service names (database:5432)

Level 2: Adding Redis and Environment Files

yaml
version: '3.8'

services:
  web:
    build:
      # Why: Build from Dockerfile in current directory
      context: .
      # Why: Specify which Dockerfile to use
      dockerfile: Dockerfile
      # Why: Pass build arguments
      args:
        - NODE_VERSION=18
    ports:
      - "3000:3000"
    # Why: Load environment variables from file
    env_file:
      - .env.development
    # Why: Override specific variables
    environment:
      - REDIS_URL=redis://cache:6379
      - DB_HOST=database
    depends_on:
      - database
      - cache
    # Why: Restart container if it crashes
    restart: unless-stopped
    # Why: Connect to specific network
    networks:
      - frontend
      - backend

  database:
    image: postgres:15-alpine
    volumes:
      - db_data:/var/lib/postgresql/data
      # Why: Run initialization SQL on first start
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    env_file:
      - .env.database
    networks:
      - backend
    # Why: Healthcheck to verify database is ready
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myuser"]
      interval: 10s
      timeout: 5s
      retries: 5

  cache:
    image: redis:7-alpine
    # Why: Persist Redis data
    volumes:
      - redis_data:/data
    networks:
      - backend
    # Why: Enable Redis persistence
    command: redis-server --appendonly yes

networks:
  # Why: Isolate frontend from direct database access
  frontend:
  # Why: Backend services communicate here
  backend:

volumes:
  db_data:
  redis_data:

• Custom build from Dockerfile
• Environment files for configuration management
• Health checks ensure services are ready
• Multiple networks for security isolation
• Restart policies for resilience

Level 3: Advanced Production Ready Setup

yaml
version: '3.8'

services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      # Why: Custom nginx configuration
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      # Why: SSL certificates
      - ./nginx/certs:/etc/nginx/certs:ro
      # Why: Static files served by nginx
      - static_files:/var/www/static:ro
    depends_on:
      - web
    networks:
      - frontend
    restart: always
    # Why: Limit resources to prevent one container hogging CPU
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
        reservations:
          cpus: '0.25'
          memory: 256M

  web:
    build:
      context: .
      dockerfile: Dockerfile
      # Why: Build production optimized image
      target: production
    # Why: Scale to multiple instances
    scale: 3
    env_file:
      - .env.production
    environment:
      - NODE_ENV=production
      - DB_HOST=database
      - REDIS_URL=redis://cache:6379
    volumes:
      # Why: Shared static files between instances
      - static_files:/app/public
    depends_on:
      database:
        # Why: Wait for database to be healthy, not just started
        condition: service_healthy
      cache:
        condition: service_started
    networks:
      - frontend
      - backend
    restart: always
    # Why: Health check for load balancing
    healthcheck:
      test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    # Why: Log to file with rotation
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

  database:
    image: postgres:15-alpine
    volumes:
      - db_data:/var/lib/postgresql/data
      # Why: Custom PostgreSQL configuration
      - ./postgres/postgresql.conf:/etc/postgresql/postgresql.conf:ro
    env_file:
      - .env.database
    networks:
      - backend
    restart: always
    # Why: Set shared memory size for PostgreSQL
    shm_size: 256mb
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
      interval: 10s
      timeout: 5s
      retries: 5
    # Why: Resource limits for database
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          cpus: '1'
          memory: 1G

  cache:
    image: redis:7-alpine
    volumes:
      - redis_data:/data
      # Why: Custom Redis configuration
      - ./redis/redis.conf:/usr/local/etc/redis/redis.conf:ro
    networks:
      - backend
    restart: always
    # Why: Use custom config
    command: redis-server /usr/local/etc/redis/redis.conf
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 3

  # Why: Background worker for async tasks
  worker:
    build:
      context: .
      dockerfile: Dockerfile
      target: production
    env_file:
      - .env.production
    environment:
      - NODE_ENV=production
      - DB_HOST=database
      - REDIS_URL=redis://cache:6379
    depends_on:
      - database
      - cache
    networks:
      - backend
    restart: always
    # Why: Worker doesn't need a port
    command: npm run worker

  # Why: Database backup service
  backup:
    image: postgres:15-alpine
    volumes:
      - ./backups:/backups
      - db_data:/var/lib/postgresql/data:ro
    env_file:
      - .env.database
    networks:
      - backend
    # Why: Run backup script daily
    entrypoint: /bin/sh -c "while true; do pg_dump -U ${POSTGRES_USER} ${POSTGRES_DB} > /backups/backup_$$(date +%Y%m%d_%H%M%S).sql; sleep 86400; done"

networks:
  frontend:
    # Why: Use custom subnet
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/16
  backend:
    # Why: Internal network not exposed
    driver: bridge
    internal: true

volumes:
  db_data:
    # Why: Use local driver with specific options
    driver: local
  redis_data:
    driver: local
  static_files:
    driver: local

• Nginx reverse proxy for SSL termination and load balancing
• Scaling multiple web instances
• Health checks with custom conditions
• Resource limits preventing resource starvation
• Logging configuration with rotation
• Background workers for async processing
• Automated backups running on schedule
• Custom networks with subnets
• Volume drivers for advanced storage

Essential Docker Compose Commands

bash
# Start all services
docker compose up
docker compose up -d  # Detached mode

# Start specific services
docker compose up web database

# Stop all services
docker compose down
docker compose down -v  # Also remove volumes
docker compose down --rmi all  # Remove images too

# View running services
docker compose ps

# View logs
docker compose logs
docker compose logs -f  # Follow logs
docker compose logs web  # Specific service

# Build or rebuild services
docker compose build
docker compose build --no-cache

# Restart services
docker compose restart
docker compose restart web

# Scale services
docker compose up -d --scale web=5

# Execute command in service
docker compose exec web bash
docker compose exec database psql -U myuser

# View service configuration
docker compose config

# Pull latest images
docker compose pull

# Pause/unpause services
docker compose pause
docker compose unpause

# Stop services without removing
docker compose stop
docker compose start

# Run one-off command
docker compose run web npm test
docker compose run --rm web npm install  # Remove container after

# View resource usage
docker compose top

Advanced Compose Techniques

Using Extends for Configuration Reuse

yaml
version: '3.8'

services:
  web:
    build: .
    environment:
      - NODE_ENV=development

yaml
version: '3.8'

services:
  web:
    ports:
      - "3000:3000"
    volumes:
      - ./app:/app

Multiple Compose Files

bash
# Development
docker compose -f docker-compose.yml -f docker-compose.dev.yml up

# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml up

Environment Variable Substitution

yaml
services:
  web:
    image: nginx:${NGINX_VERSION:-latest}
    ports:
      - "${WEB_PORT:-80}:80"

NGINX_VERSION=1.25
WEB_PORT=8080

Common Pitfalls and Solutions

yaml
#  WRONG: depends_on only waits for container start
services:
  web:
    depends_on:
      - database

#  RIGHT: Use health checks
services:
  web:
    depends_on:
      database:
        condition: service_healthy
  database:
    healthcheck:
      test: ["CMD", "pg_isready"]

Compose vs Kubernetes

• Developing locally
• Running on a single host
• Need simple orchestration
• Testing microservices

• Running in production at scale
• Need multi-host clustering
• Require auto-scaling and self-healing
• Managing hundreds of containers

Mastering Multi-Container Applications

Next Steps

Resources

• Docker Compose Documentation
• Compose File Reference
• Awesome Compose Examples