🎬

Prefer video? Watch Docker Layers Explained - Why 90% Get This Wrong on YouTube (6 min)

What Are Docker Image Layers?

Docker images are not monolithic files but rather collections of read-only filesystem layers stacked on top of each other. Each layer represents a set of filesystem changes - additions, deletions, or modifications - from the layer below it.

A Docker image layer is a read-only filesystem change that gets stacked on top of previous layers to form a complete image. Each layer contains only the differences from the layer below it, making Docker images efficient to store and transfer.

The Layer Stack Architecture

When you build a Docker image, each instruction in your Dockerfile creates a new layer:

Layer 5: Application source code (COPY . .)
Layer 4: Application dependencies (RUN pip install -r requirements.txt)
Layer 3: Python runtime (FROM python:3.11-slim)
Layer 2: Debian slim base libraries
Layer 1: Base operating system filesystem

This layered approach provides several critical benefits:

  • Efficient Storage: Multiple containers from the same image share read-only layers, dramatically reducing disk usage
  • Faster Deployments: Only changed layers need to be transferred when updating images
  • Build Caching: Unchanged layers are reused, speeding up subsequent builds
  • Incremental Updates: Changes are isolated to specific layers

Read-Only vs. Writable Layers

Image layers are immutable - once created, they cannot be modified. When you run a container from an image, Docker adds a thin writable layer (also called the "container layer") on top of the image layers.

All changes made to the running container - new files, modified files, deleted files - are written to this container layer. When the container is removed, this writable layer is discarded, leaving the original image layers unchanged.

DCA Exam Tip: The exam tests your understanding of where layers reside on the filesystem. On Linux, Docker stores layers at /var/lib/docker/overlay2/ (overlay2 driver) or /var/lib/containerd/ (containerd).

Union Filesystem and Copy-on-Write

Docker's layer system is powered by union filesystems - a technology that presents multiple separate directories as a single unified view.

OverlayFS: Docker's Storage Foundation

As of Docker 25.x, the overlay2 storage driver was the default for most Linux distributions. It uses OverlayFS, a union filesystem built into the Linux kernel.

OverlayFS works with two main directories:

  • lowerdir: Read-only layers (the image layers)
  • upperdir: Read-write layer (the container layer)
Container sees:     /app/main.py, /app/config.json, /lib/python3.11...
                            |
                    [Union Mount]
                            |
    UpperDir (RW)   +   LowerDir (RO)   +   LowerDir (RO)   + ...
    (container)         (app layer)         (python layer)

Docker Engine 29.0: The containerd Shift

Important Update (Late 2024): Docker Engine 29.0 and later uses the containerd image store by default for fresh installations. This replaces the overlay2 storage driver with containerd snapshotters.

While the underlying implementation differs, the conceptual model of stacked layers remains the same. The key changes are:

  • Improved image management and compatibility with OCI standards
  • Better integration with containerd for orchestration platforms
  • Maintained backward compatibility with existing Dockerfiles and images

Copy-on-Write (CoW) Mechanism

Docker uses a Copy-on-Write (CoW) strategy that enables containers to share base image layers while maintaining independent writable layers, reducing disk usage by up to 95%.

The Copy-on-Write strategy is central to Docker's efficiency:

  1. When a container needs to read a file, the filesystem traverses layers from top to bottom until finding it
  2. When a container needs to modify a file from a lower layer:
    • The entire file is copied to the writable container layer
    • Modifications are made to this copy
    • Subsequent reads/writes use the copied version

This approach means:

  • Containers start in milliseconds (no filesystem to copy)
  • Multiple containers share the same base image layers efficiently
  • The writable layer remains as small as possible
Performance Consideration: The first write to a file from a lower layer incurs latency (the "copy-up" operation). This is negligible for small files but can be noticeable for large files like databases. This is why Docker volumes are recommended for write-intensive workloads.

Which Dockerfile Instructions Create Layers?

Not all Dockerfile instructions create filesystem layers. Understanding which do is essential for the DCA exam and for image optimization.

Instructions That Create Filesystem Layers

Only three Dockerfile instructions create filesystem layers with actual size: RUN, COPY, and ADD. All other instructions like ENV, LABEL, and EXPOSE create metadata-only layers with zero size.

Instruction Creates Layer Example
RUN Yes (filesystem changes) RUN apt-get update && apt-get install -y curl
COPY Yes (adds files) COPY requirements.txt /app/
ADD Yes (adds files, extracts archives) ADD app.tar.gz /app/

Instructions That Create Metadata-Only Layers

These instructions create layers with 0 bytes - they only add metadata:

Instruction Purpose Example
FROM Sets base image FROM python:3.11-slim
ENV Sets environment variables ENV NODE_ENV=production
LABEL Adds metadata labels LABEL version="1.0"
EXPOSE Documents ports EXPOSE 8080
WORKDIR Sets working directory WORKDIR /app
USER Sets runtime user USER appuser
CMD Sets default command CMD ["python", "app.py"]
ENTRYPOINT Sets entry point ENTRYPOINT ["python"]

Layer Creation Example

Given this Dockerfile:

FROM python:3.11-slim          # Metadata (uses base image layers)
WORKDIR /app                   # Metadata (0 bytes)
ENV PYTHONUNBUFFERED=1         # Metadata (0 bytes)
COPY requirements.txt .        # Layer 1: ~1KB
RUN pip install -r requirements.txt  # Layer 2: ~50MB
COPY . .                       # Layer 3: ~5MB
EXPOSE 8000                    # Metadata (0 bytes)
CMD ["python", "manage.py", "runserver"]  # Metadata (0 bytes)

This creates only 3 filesystem layers on top of the base image layers.

Layer Caching: The 80% Build Time Reduction Trick

Docker's build cache is one of its most powerful features for development productivity. Teams have reported reducing build times from 18 minutes to 3 minutes simply by understanding layer caching.

How Build Caching Works

Docker's layer caching follows a simple rule: once a layer changes, all downstream layers must be rebuilt. Order your Dockerfile from least to most frequently changing instructions.

  1. For each instruction, Docker generates a cache key based on:
    • The instruction itself
    • The parent layer's cache key
    • For COPY/ADD: checksums of the source files
  2. If a cache key matches an existing layer, Docker reuses it
  3. Critical Rule: Once a layer's cache is invalidated, all subsequent layers must be rebuilt

Cache Invalidation Cascade

FROM node:20-alpine           # Cached
WORKDIR /app                  # Cached
COPY package*.json ./         # Cached (if package.json unchanged)
RUN npm install               # Cached (if package.json unchanged)
COPY . .                      # INVALIDATED (source code changed)
RUN npm run build             # MUST REBUILD (downstream)
CMD ["node", "dist/main.js"]  # MUST REBUILD (downstream)

Optimizing for Cache Efficiency

Order by change frequency - put rarely-changing instructions first:

# GOOD: Dependencies before source code
COPY package.json package-lock.json ./
RUN npm ci
COPY . .

# BAD: Source code before dependencies (invalidates cache on every code change)
COPY . .
RUN npm ci

BuildKit Cache Features

Docker BuildKit (enabled by default in recent versions) provides advanced caching:

# Cache mounts for package managers
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

# Bind mounts for build-only files (don't create layers)
RUN --mount=type=bind,source=src,target=/src \
    gcc /src/main.c -o /app/main
Real-world Impact: One team reduced their Docker build times from 18 minutes to 3 minutes (83% reduction) simply by reordering Dockerfile instructions to maximize cache hits.

Inspecting Layers: docker history and Dive

docker history Command

The Dive tool analyzes Docker images layer by layer, showing exactly which files were added, modified, or removed in each layer.

The built-in docker history command shows layer information:

# Basic usage
docker history nginx:latest

# Full output without truncation
docker history --no-trunc nginx:latest

# Custom format for specific fields
docker history --format "table {{.ID}}\t{{.Size}}\t{{.CreatedBy}}" nginx:latest

# Quiet mode (just layer IDs)
docker history -q nginx:latest

Output explanation:

  • IMAGE: Layer ID (truncated SHA256)
  • CREATED: When the layer was created
  • CREATED BY: The Dockerfile instruction that created it
  • SIZE: Layer size (layers with 0B are metadata-only)

docker inspect for Layer Details

# Get layer digests
docker inspect nginx:latest --format '{{.RootFS.Layers}}'

# Full inspection with layer information
docker inspect nginx:latest | jq '.[0].RootFS'

Dive: Advanced Layer Analysis

Dive is an open-source tool that provides interactive layer exploration:

Installation:

# macOS
brew install dive

# Ubuntu/Debian
wget https://github.com/wagoodman/dive/releases/download/v0.12.0/dive_0.12.0_linux_amd64.deb
sudo dpkg -i dive_0.12.0_linux_amd64.deb

# Docker (no installation)
docker run -ti --rm -v /var/run/docker.sock:/var/run/docker.sock wagoodman/dive nginx:latest

Key Dive features:

  • Layer-by-layer file tree navigation
  • Files added/modified/removed in each layer highlighted
  • Image efficiency score and wasted space metrics
  • CI/CD integration for automated checks

CI/CD Usage:

CI=true dive nginx:latest

With a .dive-ci configuration file:

rules:
  lowestEfficiency: 0.95
  highestWastedBytes: 20MB
  highestUserWastedPercent: 0.20

SlimToolkit (formerly DockerSlim)

SlimToolkit automatically optimizes images by analyzing what files are actually used:

# Basic minification
slim build nginx:latest

# With probe commands for dynamic analysis
slim build --http-probe nginx:latest

SlimToolkit can reduce image sizes by up to 30x by removing unused files, libraries, and dependencies.

Optimization Strategies for Smaller Images

Multi-stage builds can reduce image sizes by 90% or more by separating build-time dependencies from the final runtime image.

Strategy 1: Minimize Number of Layers

Combine RUN instructions:

# BAD: 3 layers
RUN apt-get update
RUN apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*

# GOOD: 1 layer
RUN apt-get update && \
    apt-get install -y curl && \
    rm -rf /var/lib/apt/lists/*

Strategy 2: Multi-Stage Builds

Multi-stage builds separate build-time dependencies from runtime:

# Stage 1: Build
FROM node:20 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: Production
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
CMD ["node", "dist/main.js"]

Results: A 1GB Node.js image can be reduced to under 100MB.

Strategy 3: Use Minimal Base Images

Base Image Size Use Case
ubuntu:22.04 ~77MB Full-featured, familiar
debian:bookworm-slim ~74MB Debian minimal
alpine:3.19 ~7MB Ultra-minimal, musl libc
distroless ~2MB No shell, maximum security
scratch 0MB Static binaries only

Strategy 4: Proper .dockerignore

A well-structured .dockerignore file can reduce build context from 2GB to 50MB, dramatically speeding up builds and preventing sensitive files from entering images.

# Version control
.git
.gitignore

# Dependencies (will be installed in image)
node_modules
__pycache__
*.pyc
venv/

# Development files
.env
.env.*
*.log
.DS_Store

# IDE
.idea/
.vscode/
*.swp

# Test and docs
tests/
docs/
*.md
!README.md

# Build artifacts
dist/
build/
*.egg-info/

Strategy 5: Clean Up in Same Layer

# BAD: Deleted files still in previous layer
RUN apt-get update && apt-get install -y build-essential
RUN make && make install
RUN apt-get remove -y build-essential  # Still takes space!

# GOOD: Clean up in same layer
RUN apt-get update && \
    apt-get install -y build-essential && \
    make && make install && \
    apt-get remove -y build-essential && \
    apt-get autoremove -y && \
    rm -rf /var/lib/apt/lists/*

Strategy 6: Layer Squashing

Layer squashing combines multiple layers into one, reducing image size when files are created and deleted across layers, but eliminates caching benefits.

# Using Docker's experimental squash flag
docker build --squash -t myimage:latest .

When to squash:

  • Files are created and deleted across layers
  • Before pushing to a registry
  • When caching benefits are not needed

When NOT to squash:

  • During development (lose caching)
  • When sharing base layers with other images

Common Mistakes and How to Avoid Them

Mistake 1: Wrong Instruction Order

# WRONG: Source code changes invalidate dependency cache
COPY . .
RUN pip install -r requirements.txt

# RIGHT: Dependencies first, then source
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

Mistake 2: Using latest Tag

# RISKY: Non-reproducible builds
FROM python:latest

# SAFE: Pinned version
FROM python:3.11.7-slim-bookworm

Mistake 3: Not Cleaning Package Caches

# BAD: Cache takes space
RUN apt-get update && apt-get install -y curl

# GOOD: Remove cache in same layer
RUN apt-get update && \
    apt-get install -y curl && \
    rm -rf /var/lib/apt/lists/*

Mistake 4: Missing .dockerignore

Without .dockerignore:

  • .git folder (potentially 100s of MB) enters build context
  • node_modules gets copied (then reinstalled anyway)
  • Sensitive files like .env may enter the image

Mistake 5: Running as Root

# INSECURE: Running as root
CMD ["python", "app.py"]

# SECURE: Create and use non-root user
RUN useradd -m -s /bin/bash appuser
USER appuser
CMD ["python", "app.py"]

Mistake 6: Storing Secrets in Layers

# DANGEROUS: Secret visible in layer history
COPY .env /app/
RUN source /app/.env && ./configure
RUN rm /app/.env  # Still visible in previous layer!

# SAFE: Use build secrets (BuildKit)
RUN --mount=type=secret,id=mysecret,target=/app/.env \
    source /app/.env && ./configure

Debugging Layer Issues

# Verbose build output
docker build --progress=plain .

# Build up to specific stage
docker build --target=builder .

# Run shell in intermediate layer
docker run -it <layer-id> sh

# Check which files are in build context
docker build -t test . 2>&1 | head -20

DCA Exam Preparation: Layer Questions

Exam Weight: Domain 2 (Image Creation, Management, and Registry) accounts for 20% of the DCA exam. Layer-related questions are heavily tested.

Domain 2 Coverage

The DCA exam tests the following layer-related competencies:

  1. Display layers of a Docker image
    • docker history <image>
    • docker inspect <image> --format '{{.RootFS.Layers}}'
  2. Understand layer composition
    • Know which instructions create layers (RUN, COPY, ADD)
    • Understand where layers reside (/var/lib/docker/overlay2/)
  3. Modify image to single layer
    • Multi-stage builds
    • docker build --squash
    • Export/import (docker export / docker import)
  4. Create efficient images
    • Layer ordering for cache optimization
    • Combining RUN instructions
    • Using .dockerignore
    • Minimal base images

Practice Commands for DCA

# Display image layers
docker history nginx:latest

# Show layer details
docker history --no-trunc nginx:latest

# Get layer digests
docker inspect nginx:latest --format '{{json .RootFS.Layers}}'

# Check image size breakdown
docker system df -v

# View storage driver info
docker info | grep -i storage

# Export container filesystem (flattens layers)
docker export mycontainer > container.tar

# Import as single-layer image
docker import container.tar mynewimage:latest

Sample Exam Questions

Q: Which Dockerfile instruction does NOT create a filesystem layer?

  • RUN
  • COPY
  • ENV
  • ADD

Answer: C) ENV - ENV creates metadata only

Q: How many layers can a Docker image have?

Answer: 127 layers maximum

Q: What happens when you modify a file from a lower layer in a running container?

Answer: Docker uses copy-on-write to copy the file to the writable container layer before modification

Q: How do you display the layers of a Docker image?

Answer: docker history <image>

Official DCA Resources

Frequently Asked Questions

What is a Docker image layer?

A Docker image layer is a read-only filesystem change that gets stacked on top of previous layers to form a complete image. Each layer contains only the differences from the layer below it, making Docker images efficient to store and transfer. Layers are created by RUN, COPY, and ADD instructions in Dockerfiles.

Which Dockerfile instructions create layers?

Only three Dockerfile instructions create filesystem layers with actual size: RUN, COPY, and ADD. Other instructions like ENV, LABEL, EXPOSE, WORKDIR, USER, CMD, and ENTRYPOINT create metadata-only layers with zero bytes that don't contribute to image size.

How does Docker's layer caching work?

Docker caches each layer during builds. If an instruction's inputs haven't changed, Docker reuses the cached layer. However, when one layer changes, all subsequent layers must be rebuilt. This is why you should order Dockerfile instructions from least to most frequently changing to maximize cache efficiency.

What is the maximum number of layers a Docker image can have?

Docker images can have a maximum of 127 layers, a limitation inherited from the AUFS storage driver that persists across storage backends. While this limit rarely affects modern, well-optimized images, it's important to combine RUN instructions and use multi-stage builds to stay well below this limit.

How can I reduce Docker image size using layers?

Use multi-stage builds to separate build dependencies from runtime, combine RUN commands with && to reduce layers, use minimal base images like Alpine, add a .dockerignore file to exclude unnecessary files, and clean up package manager caches in the same layer where you install packages.

What is Copy-on-Write in Docker?

Copy-on-Write (CoW) is Docker's strategy where containers share base image layers but maintain independent writable layers. When a container needs to modify a file from a lower layer, the entire file is copied to the writable container layer before modification. This enables containers to start in milliseconds and reduces disk usage by up to 95%.

How do I view Docker image layers?

Use the docker history command to view image layers: docker history nginx:latest. For more detailed analysis, use the Dive tool which provides interactive layer exploration, file tree navigation, and image efficiency metrics. You can also use docker inspect to get layer digests.

Conclusion

Docker image layers are the fundamental building blocks that make containerization efficient and practical. Understanding how layers work is essential for:

  • Passing the DCA exam: Domain 2 (20% of exam) heavily tests layer concepts
  • Optimizing build times: Proper layer ordering can reduce builds from 18 minutes to 3 minutes
  • Reducing image sizes: Multi-stage builds can achieve 90% size reductions
  • Improving deployment speed: Smaller images with shared layers transfer faster

The key takeaways to remember:

  1. Only RUN, COPY, and ADD create filesystem layers
  2. Layer caching invalidates downstream when any layer changes
  3. Copy-on-Write enables containers to share image layers efficiently
  4. Multi-stage builds are the most powerful optimization technique
  5. Docker Engine 29.0+ uses containerd by default, but layer concepts remain consistent

Next Steps

Ready to Master Docker for the DCA Exam?

Subscribe to our YouTube channel for hands-on Docker tutorials, DCA exam prep, and DevOps best practices.

Subscribe to Gheware DevOps AI