Why containers for AI apps

AI apps are dependency-heavy and environment-sensitive: specific Python and library versions, system packages, model SDKs, sometimes native builds. "Works on my machine" fails hard here. A container packages your app and its entire environment into one image that runs identically on your laptop, a teammate's machine, a VPS, or Azure — the reproducibility LLMOps (Track 5) demands, made concrete. Containers are the unit every modern cloud deploy target consumes (Azure Container Apps, Kubernetes, Cloud Run), so this is the foundation for the rest of the track.

Vocabulary: a Dockerfile is the recipe; building it produces an image (an immutable snapshot); running an image gives a container (a live instance). One image → many identical containers.

A Dockerfile for a FastAPI LLM service

Here's a working Dockerfile for the kind of FastAPI service you built in Track 1 (and used as the hybrid code-service in Track 4), annotated line by line:

FROM python:3.12-slim                     # base image: a minimal Python 3.12 (slim = no build tools/docs, smaller)

WORKDIR /app                              # set the working dir inside the container (like `cd /app`)

COPY requirements.txt .                   # copy ONLY deps first (this line's cache survives code changes)
RUN pip install --no-cache-dir -r requirements.txt   # install deps; --no-cache-dir keeps the image smaller

COPY . .                                  # now copy the app code (changes here don't bust the deps layer above)

EXPOSE 8000                               # document the port the app listens on
ENV PORT=8000                             # default config via env (overridable at run time)

CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]   # the start command (like `node server.js`)

The order matters for caching: each instruction is a layer, and Docker reuses cached layers until one changes. Copying requirements.txt and installing deps before copying your code means editing a .py file doesn't reinstall dependencies — a huge build-time saving. Copy code last; copy rarely-changing things first.

Keep images small

Image size affects push/pull time, cold starts, and cost. Two levers:

Slim/minimal base images. python:3.12-slim is far smaller than the full python:3.12; avoid the giant default unless you need its build tools. Also add a .dockerignore (like .gitignore) so you don't copy node_modules-equivalents, .git, data files, and caches into the image.

Multi-stage builds. Build in one stage (with compilers/dev tools), then copy only the finished artifacts into a clean, tiny runtime stage — so build-time bloat never ships:

# ── Stage 1: build (has compilers to build any native wheels) ──
FROM python:3.12 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --target=/deps -r requirements.txt   # install into a folder we can copy out

# ── Stage 2: runtime (slim; only the finished deps + code) ──
FROM python:3.12-slim                       # a clean, small final image
WORKDIR /app
COPY --from=builder /deps /usr/local/lib/python3.12/site-packages   # copy ONLY the built deps from stage 1
COPY . .
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
# The compilers and build junk live only in stage 1 and never reach the shipped image.

docker-compose: the app and its dependencies

An AI app is rarely alone — it needs a vector DB, maybe Redis, maybe a worker. docker-compose defines and runs the whole set together with one command, wiring their networking for you:

# docker-compose.yml — the app plus its pgvector database
services:
  app:
    build: .                              # build the image from the Dockerfile in this dir
    ports: ["8000:8000"]                  # map host:container port
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}  # pass a secret from the HOST env — NOT hardcoded here
      - DATABASE_URL=postgresql://db/rag  # `db` resolves to the db service below (compose networking)
    depends_on: [db]                      # start db first
  db:
    image: pgvector/pgvector:pg16         # the vector DB from Track 3, as a ready-made image
    environment:
      - POSTGRES_DB=rag
    volumes: ["pgdata:/var/lib/postgresql/data"]   # persist data outside the container (survives restarts)
volumes:
  pgdata:

docker compose up builds and starts both, on a shared network where app reaches the database at host db. This is your whole stack, reproducible, in one file — the local mirror of what you'll deploy to the cloud.

Config and secrets by environment

Never bake secrets (API keys, DB passwords) or environment-specific config into the image — an image is shared and often pushed to a registry, so a baked-in key is a leaked key. The rule (the twelve-factor principle): config comes from the environment at run time.

import os
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]     # read from env at runtime (like process.env.X)
DATABASE_URL   = os.environ.get("DATABASE_URL", "postgresql://localhost/rag")   # with a sensible default
# The SAME image runs in dev, staging, prod — only the injected env differs. No rebuild to change config.

At run time the platform injects them: docker run -e OPENAI_API_KEY=..., compose's environment:, or (in the cloud) a managed secret store (Azure Key Vault, next topics). This is also what makes the LLMOps promote/ rollback flow clean — the same immutable image, different injected config per environment.

Pitfalls

• Fat base images. Defaulting to full python:3.12 bloats size and cold starts — use -slim and multi-stage.
• Bad layer order. Copying code before installing deps busts the dependency cache on every code change.
• Secrets baked into the image. A key in the Dockerfile/code/build context ships to anyone with the image — inject at run time.
• No .dockerignore. Copying .git, data, and caches bloats the image and can leak files.
• Non-reproducible builds. Unpinned dependency versions make images drift — pin requirements.txt (LLMOps reproducibility).

Recap

• A container packages the app + its whole environment into an image that runs identically everywhere — the reproducibility deploys need and the unit clouds consume.
• Order Dockerfile instructions for layer caching (deps before code); keep images small with slim bases, .dockerignore, and multi-stage builds.
• Use docker-compose to run the app with its dependencies (vector DB, etc.) as one reproducible stack.
• Pass config and secrets by environment at run time — never bake them into the image; the same image runs in every environment.
• Worked examples map to the exercises: a Dockerfile (Ex 1), multi-stage slimming (Ex 2), compose with pgvector (Ex 3), env-based config (Ex 4).

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

docker build -t my-llm-app .
docker run -p 8000:8000 -e OPENAI_API_KEY=$OPENAI_API_KEY my-llm-app
curl localhost:8000/health
# edit app.py, then:
docker build -t my-llm-app .        # note: "Using cache" on the pip install layer — deps not reinstalled

FROM python:3.12 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --target=/deps -r requirements.txt

FROM python:3.12-slim
WORKDIR /app
COPY --from=builder /deps /usr/local/lib/python3.12/site-packages
COPY . .
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

docker images my-llm-app    # compare the SIZE column vs the single-stage build

services:
  app:
    build: .
    ports: ["8000:8000"]
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - DATABASE_URL=postgresql://postgres@db:5432/rag
    depends_on: [db]
  db:
    image: pgvector/pgvector:pg16
    environment: [POSTGRES_DB=rag, POSTGRES_HOST_AUTH_METHOD=trust]
    volumes: ["pgdata:/var/lib/postgresql/data"]
volumes:
  pgdata:

docker compose up

import os
API_KEY = os.environ["OPENAI_API_KEY"]
DB_URL  = os.environ.get("DATABASE_URL", "postgresql://localhost/rag")
LOG_LEVEL = os.environ.get("LOG_LEVEL", "info")

# dev
docker run -e OPENAI_API_KEY=$DEV_KEY  -e LOG_LEVEL=debug my-llm-app
# mock prod — same image, different injected config, no rebuild
docker run -e OPENAI_API_KEY=$PROD_KEY -e LOG_LEVEL=warning -e DATABASE_URL=$PROD_DB my-llm-app