Virtualization8 min read

Building Images (Dockerfile & BuildKit)

Dockerfile mechanics, layer caching, multi-stage builds, and BuildKit

Dockerfile Mechanics

A Dockerfile is a sequence of instructions that produce an OCI image. Each instruction is executed in order, and some create new filesystem layers while others only modify image metadata.

Layer-Creating Instructions

These instructions modify the filesystem and produce a new layer in the image:

Instruction	What It Does
`RUN`	Executes a command in a temporary container, snapshots the filesystem changes
`COPY`	Copies files from build context into the image filesystem
`ADD`	Like COPY, but also auto-extracts tar archives and supports URLs (prefer COPY)

Metadata-Only Instructions

These modify the image config JSON but do not create filesystem layers:

Instruction	What It Sets
`ENV`	Environment variables in config
`EXPOSE`	Documents ports (does NOT publish them)
`LABEL`	Key-value metadata on the image
`CMD`	Default arguments to entrypoint
`ENTRYPOINT`	The executable to run
`WORKDIR`	Sets working directory for subsequent instructions
`USER`	Sets UID/GID for RUN, CMD, ENTRYPOINT

The Build Context

What Gets Sent to the Daemon

When you run docker build ., the entire directory (the build context) is tar'd and sent to the Docker daemon. This happens before any Dockerfile instruction runs.

Warning

Without a .dockerignore, everything is sent — including .git/ (potentially hundreds of MB), node_modules/, build artifacts, secrets, etc. This slows down builds and risks leaking sensitive files.

No .dockerignore

console


          $ docker build .

          Sending build context: 847MB

          # .git/ alone is 200MB

          # node_modules/ is 500MB

          # Entire build takes 45 seconds

With .dockerignore

console


          $ cat .dockerignore

          .git
node_modules
*.log
dist/
.env*


          $ docker build .

          Sending build context: 2.3MB

          # Build takes 3 seconds

Layer Caching Rules

Docker's build cache is the primary mechanism for fast incremental builds. Understanding the cache invalidation rules is critical for writing efficient Dockerfiles.

How the Cache Works

Docker processes instructions top-down, one at a time
For each instruction, it checks: same instruction text + same parent layer? If yes → cache hit, reuse the cached layer
For COPY / ADD: cache is based on file content checksums, not timestamps. Changed file content → cache miss
Any cache miss invalidates ALL subsequent layers. Every instruction after the first miss must be re-executed, even if they haven't changed

Warning

The cascade rule: Once the cache is busted at instruction N, instructions N+1, N+2, ... all rebuild from scratch. This is why instruction ordering matters enormously.

Instruction Ordering: The Critical Optimization

BAD — deps reinstalled on every code change

console

FROM node:20-alpine

WORKDIR /app

# Copies EVERYTHING — including source code
COPY . .            # cache busts when ANY file changes

RUN npm install     # forced to re-run every time!
RUN npm run build   # also re-runs every time

CMD ["node", "dist/server.js"]

Changed one line of code? Full npm install runs again (minutes).

GOOD — deps cached, only code rebuilds

console

FROM node:20-alpine

WORKDIR /app

# Copy ONLY dependency files first
COPY package.json package-lock.json ./

# Install deps — cached unless package*.json changed
RUN npm ci

# NOW copy source code
COPY . .            # cache busts here on code change
RUN npm run build   # only this re-runs

CMD ["node", "dist/server.js"]

Changed one line of code? npm ci is cached (seconds).

Tip

Rule of thumb: Order Dockerfile instructions from least frequently changing (top) to most frequently changing (bottom). Base image → system packages → dependency manifests → dependency install → source code → build.

Multi-Stage Builds

Multi-stage builds use multiple FROM instructions in a single Dockerfile. Each FROM starts a new stage with a fresh filesystem. You can copy artifacts between stages using COPY --from=. Only the final stage becomes the output image.

Use Case: Build in Full SDK, Deploy Minimal Runtime

console

# ---- Stage 1: Build ----
FROM golang:1.21 AS builder
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /app/server ./cmd/server

# ---- Stage 2: Runtime ----
FROM alpine:3.18
RUN apk --no-cache add ca-certificates
COPY --from=builder /app/server /app/server
USER 1000:1000
CMD ["/app/server"]

Single-Stage Build

Final image contains:

Go compiler + toolchain (~400 MB)
All source code
All build dependencies
The compiled binary

Image size: ~1.1 GB

Multi-Stage Build

Final image contains:

Alpine base (~5 MB)
CA certificates (~1 MB)
The compiled binary (~10 MB)
Nothing else

Image size: ~16 MB

Note

Named stages: Use AS builder to name stages for readability. You can also reference stages by index: COPY --from=0 copies from the first FROM. Named stages are strongly preferred.

BuildKit

BuildKit is the next-generation image builder, the default since Docker 23.0. It replaces the legacy builder with a fundamentally different architecture.

Key Improvements Over Legacy Builder

Parallel Stage Building

BuildKit analyzes the Dockerfile as a DAG (directed acyclic graph). Independent stages build concurrently. If stage A and stage B don't depend on each other, they run in parallel.

Stage A
Build frontend
Stage B
Build backend
Stage C
COPY --from A,B

Stages A and B run concurrently; C waits for both.

LLB (Low-Level Build)

BuildKit doesn't execute Dockerfile instructions directly. It compiles the Dockerfile into LLB — an intermediate representation that is a DAG of build operations.

Enables optimizations (dead code elimination, parallel execution)
Alternative frontends can generate LLB (not just Dockerfiles)
Operations are content-addressable and cacheable independently

Cache Mounts

Persist package manager caches between builds. The cache directory is mounted into the RUN instruction but is not included in the final layer.

Python (pip)

console

# syntax=docker/dockerfile:1
FROM python:3.12-slim

COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

pip cache persists between builds. Adding one new dependency doesn't redownload all the others.

Go (modules)

console

# syntax=docker/dockerfile:1
FROM golang:1.21

COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    go mod download

Both the module cache and build cache persist across builds.

Secret Mounts

Make secrets available during build without baking them into any image layer.

Example: Private NPM Registry

console

# syntax=docker/dockerfile:1
FROM node:20-alpine

WORKDIR /app
COPY package.json package-lock.json ./

# .npmrc is mounted at build time but NOT stored in the layer
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci --production

COPY . .
CMD ["node", "server.js"]

console


      # Build with the secret:

      $ docker build --secret id=npmrc,src=$HOME/.npmrc -t myapp .


      # The .npmrc is NEVER in any layer — even if you docker history the image

Warning

Legacy builder had no secret support. People used to COPY secrets in, run the command, then RUN rm the secret — but the secret was still in the layer history. Secret mounts solve this properly.

Cache Export/Import for CI

CI Cache Strategy

console

# Export cache to a registry (end of CI build):
docker build \
  --cache-to type=registry,ref=myrepo/myapp:buildcache,mode=max \
  -t myrepo/myapp:latest .

# Import cache from registry (start of next CI build):
docker build \
  --cache-from type=registry,ref=myrepo/myapp:buildcache \
  -t myrepo/myapp:latest .

mode=max exports all layers (including intermediate stages), not just the final image layers.

Dockerfile Syntax Directive

Note

Always add this as the first line: # syntax=docker/dockerfile:1
This tells BuildKit which Dockerfile parser to use, enabling features like --mount. Without it, some BuildKit features won't work. This line must be the very first line of the file (before any comments or blank lines).

Image Best Practices

Base Image Selection

Base Image	Size	Shell	Pkg Manager	libc	When to Use
`scratch`	0 B	No	No	None	Statically compiled binaries (Go with `CGO_ENABLED=0`)
`distroless`	~2 MB	No	No	glibc	Dynamic binaries that need libc but no shell (Java, Python, Node)
`alpine`	~5 MB	Yes (ash)	apk	musl	General purpose minimal. Watch for musl compatibility issues (DNS, locale)
`debian-slim`	~30 MB	Yes (bash)	apt	glibc	When you need glibc compat + shell + apt. Good default for most apps
`ubuntu`	~30 MB	Yes (bash)	apt	glibc	When you need universe/multiverse packages or LTS support

Reducing Image Size

Use multi-stage builds — build tools stay in the builder stage

Chain RUN commands to reduce layers:

yaml

RUN apt-get update \
    && apt-get install -y --no-install-recommends curl \
    && rm -rf /var/lib/apt/lists/*

Clean up in the same layer as the install (otherwise the files exist in the lower layer forever)
Use .dockerignore to exclude build artifacts, tests, docs
Use --no-install-recommends with apt-get

Don't

Install debug tools in production images (curl, vim, strace)

Leave package manager caches in the image:

console

# BAD: cache is in layer 1, rm is in layer 2
# layer 1 still contains the cache!
RUN apt-get update && apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*

Use ADD for remote URLs (use RUN curl instead — more explicit, cacheable)
Run as root (use USER 1000:1000)

Non-Root User

Running as Non-Root

console

FROM node:20-alpine

# Create a non-root user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup

WORKDIR /app
COPY --chown=appuser:appgroup . .

# Switch to non-root before CMD
USER appuser

CMD ["node", "server.js"]

Tip

Why non-root matters: If an attacker escapes the container, they land on the host with the same UID. Running as root inside the container means root on the host (unless user namespaces are configured). Running as UID 1000 limits the blast radius.

Layer Count vs. Layer Size

Trade-offs

Fewer layers (chain RUN commands)

Smaller image (cleanup happens in same layer)
Fewer metadata entries
Faster push/pull (fewer HTTP requests)
Worse caching — one change rebuilds the entire chained command

More layers (separate RUN commands)

Better cache granularity
Faster iterative development
Potentially larger image (can't clean up previous layer's files)
More pull requests to registry

In practice: group logically related operations (e.g., apt install + cleanup) but keep distinct phases separate (deps install vs. source copy).

Dockerfile Mechanics

The Build Context

Layer Caching Rules

Instruction Ordering: The Critical Optimization

Multi-Stage Builds

BuildKit

Key Improvements Over Legacy Builder

Stage A

Stage B

Stage C

Cache Mounts

Secret Mounts

Cache Export/Import for CI

Dockerfile Syntax Directive

Image Best Practices

Base Image Selection

Reducing Image Size

Non-Root User

Layer Count vs. Layer Size