Docker & Containerization Guide

This is a collection of notes covering Docker containerization fundamentals and advanced optimization techniques.

Docker Fundamentals

What is Docker?

Docker is a platform for developing, shipping, and running applications using containerization. Containers package applications with their dependencies, ensuring consistency across different environments.

Key Concepts

Image: Read-only template used to create containers
Container: Running instance of an image
Dockerfile: Text file with instructions to build an image
Registry: Storage for Docker images (Docker Hub, ECR, etc.)
Volume: Persistent data storage for containers

Development Video Presentation

Presentation Slides

Basic Docker Commands

Image Management

# Pull image from registry
$ docker pull nginx:latest
$ docker pull ubuntu:20.04

# List local images
$ docker images
$ docker image ls

# Remove images
$ docker rmi image_name:tag
$ docker image prune  # Remove unused images

# Build image from Dockerfile
$ docker build -t my-app:latest .
$ docker build -f Dockerfile.prod -t my-app:prod .

Container Management

# Run container
$ docker run -d --name my-container nginx
$ docker run -it ubuntu:20.04 /bin/bash  # Interactive
$ docker run -p 8080:80 nginx  # Port mapping

# List containers
$ docker ps      # Running containers
$ docker ps -a   # All containers

# Container operations
$ docker start container_name
$ docker stop container_name
$ docker restart container_name
$ docker rm container_name

# Execute commands in running container
$ docker exec -it container_name /bin/bash
$ docker exec container_name ls -la

Logs and Debugging

# View container logs
$ docker logs container_name
$ docker logs -f container_name  # Follow logs

# Container resource usage
$ docker stats
$ docker stats container_name

# Inspect container
$ docker inspect container_name

Dockerfile Best Practices

Basic Dockerfile Structure

# Use specific version tags
FROM node:16-alpine

# Set working directory
WORKDIR /app

# Copy package files first (leverage caching)
COPY package*.json ./

# Install dependencies
RUN npm ci --only=production

# Copy application code
COPY . .

# Create non-root user
RUN addgroup -g 1001 -S nodejs
RUN adduser -S nextjs -u 1001

# Change ownership
RUN chown -R nextjs:nodejs /app
USER nextjs

# Expose port
EXPOSE 3000

# Define entry point
CMD ["npm", "start"]

Multi-Stage Builds

Optimize image size with multi-stage builds:

# Build stage
FROM node:16-alpine AS builder

WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Production stage
FROM node:16-alpine AS production

WORKDIR /app

# Copy package files
COPY package*.json ./

# Install only production dependencies
RUN npm ci --only=production && npm cache clean --force

# Copy built application from builder stage
COPY --from=builder /app/dist ./dist

# Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nextjs -u 1001

# Change ownership
RUN chown -R nextjs:nodejs /app
USER nextjs

EXPOSE 3000
CMD ["npm", "start"]

Optimization Techniques

Layer Caching

# ❌ Poor caching - copies everything first
COPY . .
RUN npm install

# ✅ Better caching - copy package files first
COPY package*.json ./
RUN npm install
COPY . .

Reduce Image Size

# Use Alpine Linux for smaller base images
FROM node:16-alpine

# Combine RUN commands to reduce layers
RUN apt-get update && \
    apt-get install -y curl && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Remove build dependencies
RUN apk add --no-cache --virtual .build-deps \
    python3 \
    make \
    g++ && \
    npm install && \
    apk del .build-deps

Security Best Practices

# Don't run as root
RUN adduser -D -s /bin/sh appuser
USER appuser

# Use specific tags, not 'latest'
FROM node:16.14.2-alpine

# Scan for vulnerabilities
RUN npm audit fix

# Use multi-stage builds to exclude dev dependencies
FROM node:16-alpine AS dependencies
COPY package*.json ./
RUN npm ci --only=production

FROM node:16-alpine AS runtime
COPY --from=dependencies /app/node_modules ./node_modules

Docker Compose

Basic docker-compose.yml

version: '3.8'

services:
  web:
    build: .
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
    depends_on:
      - db
    volumes:
      - ./logs:/app/logs

  db:
    image: postgres:13-alpine
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  redis:
    image: redis:6-alpine
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data

volumes:
  postgres_data:
  redis_data:

Docker Compose Commands

# Start services
$ docker-compose up
$ docker-compose up -d  # Detached mode

# Stop services
$ docker-compose down
$ docker-compose down -v  # Remove volumes

# View logs
$ docker-compose logs
$ docker-compose logs -f web  # Follow specific service

# Scale services
$ docker-compose up --scale web=3

# Execute commands
$ docker-compose exec web /bin/bash
$ docker-compose run web npm test

Advanced Docker Concepts

Volumes and Data Management

Named Volumes

# Create volume
$ docker volume create my-volume

# Use volume in container
$ docker run -v my-volume:/data alpine

# List volumes
$ docker volume ls

# Inspect volume
$ docker volume inspect my-volume

Bind Mounts

# Mount host directory
$ docker run -v /host/path:/container/path alpine

# Mount current directory
$ docker run -v $(pwd):/app node:16

tmpfs Mounts

# Mount in-memory filesystem
$ docker run --tmpfs /tmp alpine

Networking

Default Networks

# List networks
$ docker network ls

# Inspect network
$ docker network inspect bridge

Custom Networks

# Create custom network
$ docker network create my-network

# Run containers on custom network
$ docker run --network my-network --name web nginx
$ docker run --network my-network --name app node:16

# Containers can communicate by name
$ docker exec app curl http://web

Health Checks

# Add health check to Dockerfile
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:3000/health || exit 1

# Health check in docker-compose.yml
services:
  web:
    build: .
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

Production Deployment

Image Registry

Docker Hub

# Tag image
$ docker tag my-app:latest username/my-app:latest

# Push to Docker Hub
$ docker push username/my-app:latest

# Pull from Docker Hub
$ docker pull username/my-app:latest

Private Registry

# Run local registry
$ docker run -d -p 5000:5000 --name registry registry:2

# Tag for private registry
$ docker tag my-app:latest localhost:5000/my-app:latest

# Push to private registry
$ docker push localhost:5000/my-app:latest

Container Orchestration

Docker Swarm

# Initialize swarm
$ docker swarm init

# Deploy stack
$ docker stack deploy -c docker-compose.yml my-stack

# List services
$ docker service ls

# Scale service
$ docker service scale my-stack_web=3

Kubernetes Integration

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: my-app:latest
        ports:
        - containerPort: 3000
        env:
        - name: NODE_ENV
          value: production

Monitoring and Logging

Container Logs

# Centralized logging with Docker
$ docker run -d \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /var/lib/docker/containers:/var/lib/docker/containers:ro \
  fluentd

Monitoring Stack

# monitoring/docker-compose.yml
version: '3.8'

services:
  prometheus:
    image: prom/prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml

  grafana:
    image: grafana/grafana
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin

  cadvisor:
    image: gcr.io/cadvisor/cadvisor
    ports:
      - "8080:8080"
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro

Performance Optimization

Image Size Optimization

Choose Minimal Base Images

# Comparison of base image sizes
FROM ubuntu:20.04      # ~72MB
FROM alpine:3.14       # ~5MB
FROM node:16-alpine    # ~110MB vs node:16 at ~900MB
FROM distroless/nodejs # ~50MB (Google distroless)

Multi-Stage Build Example

# Build stage - includes dev tools
FROM node:16 AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
RUN npm prune --production

# Production stage - minimal runtime
FROM node:16-alpine AS production
WORKDIR /app

# Copy only production dependencies
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
COPY --from=build /app/package*.json ./

USER node
EXPOSE 3000
CMD ["node", "dist/index.js"]

Runtime Performance

Resource Limits

# Limit CPU and memory
$ docker run --cpus=".5" --memory="512m" my-app

# Docker Compose
version: '3.8'
services:
  web:
    image: my-app
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
        reservations:
          cpus: '0.25'
          memory: 256M

Startup Optimization

# Optimize Node.js startup
ENV NODE_ENV=production
ENV NODE_OPTIONS="--max-old-space-size=512"

# Enable V8 optimizations
ENV NODE_OPTIONS="--optimize-for-size"

Security Best Practices

Container Security

# Use non-root user
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
USER appuser

# Use specific versions
FROM node:16.14.2-alpine

# Scan for vulnerabilities
RUN npm audit --audit-level moderate

# Remove unnecessary packages
RUN apk del .build-deps

Secrets Management

# Use Docker secrets (Swarm mode)
$ echo "mysecretpassword" | docker secret create db_password -

# Reference in docker-compose.yml
version: '3.8'
services:
  db:
    image: postgres
    secrets:
      - db_password
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password

secrets:
  db_password:
    external: true

Network Security

# Isolate services with custom networks
version: '3.8'

services:
  web:
    networks:
      - frontend
  
  api:
    networks:
      - frontend
      - backend
  
  db:
    networks:
      - backend

networks:
  frontend:
  backend:
    internal: true  # No external access

Troubleshooting

Common Issues

Container Won’t Start

# Check container logs
$ docker logs container_name

# Check exit code
$ docker ps -a

# Run in interactive mode for debugging
$ docker run -it my-app /bin/sh

Performance Issues

# Monitor resource usage
$ docker stats

# Check container processes
$ docker exec container_name ps aux

# Analyze image layers
$ docker history my-app:latest

Networking Problems

# Test connectivity between containers
$ docker exec container1 ping container2

# Check port bindings
$ docker port container_name

# Inspect network configuration
$ docker network inspect bridge

Debugging Tools

# Enter running container
$ docker exec -it container_name /bin/bash

# Copy files from container
$ docker cp container_name:/app/logs ./logs

# Create debugging container
$ docker run --rm -it --network container:target_container alpine

Development Workflow

Local Development

# Dockerfile.dev
FROM node:16-alpine

WORKDIR /app

# Install dependencies
COPY package*.json ./
RUN npm install

# Copy source code
COPY . .

# Use nodemon for development
CMD ["npm", "run", "dev"]

# docker-compose.dev.yml
version: '3.8'

services:
  web:
    build:
      context: .
      dockerfile: Dockerfile.dev
    volumes:
      - .:/app
      - /app/node_modules  # Anonymous volume for node_modules
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=development

CI/CD Integration

# .github/workflows/docker.yml
name: Docker Build and Push

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    
    - name: Build Docker image
      run: docker build -t my-app:$ .
    
    - name: Run tests
      run: docker run --rm my-app:$ npm test
    
    - name: Push to registry
      run: |
        echo $ | docker login -u $ --password-stdin
        docker push my-app:$

This comprehensive guide covers Docker fundamentals through production deployment and optimization techniques. The key is to start with basic concepts and gradually implement more advanced practices as your applications and infrastructure needs grow.