Docker builds on a 32 GB Raspberry Pi: How I recovered from a full disk

I was running a hobbyist homelab on a Raspberry Pi 4 with 32 GB of eMMC storage. The Pi was a low‑cost, low‑power server that hosted a few services: a personal web server, a GitLab runner, and a Docker build farm for CI pipelines. The build farm was a simple script that pulled the latest code, built a Docker image, and pushed it to a registry. Everything worked until the disk filled up during a large multi‑stage build.

Below is a step‑by‑step walk‑through of what happened, how I identified the root cause, and the practical fixes I applied. The goal is to give you a concrete playbook for avoiding and recovering from disk‑full scenarios on small devices.


Why the disk filled up

Docker stores layers in /var/lib/docker. On a Raspberry Pi, the default storage driver is overlay2, which keeps each layer as a separate directory. A typical multi‑stage build can generate dozens of layers, each a few megabytes, but when you add temporary build artifacts, cache directories, and the base image layers, the cumulative size can exceed the available space quickly.

In my case, the build script had a --cache-from flag that pulled a large base image from the registry. The base image itself was 1.2 GB. The build added a node_modules directory that grew to 600 MB, and the final image was 1.5 GB. The overlay filesystem kept all intermediate layers, so the total consumption hit 4 GB, which is a lot when you only have 32 GB and the OS, swap, and other services already occupy ~10 GB.


Spotting the problem early

The first sign was the df -h output:

Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   32G  28G   4G  88% /

The root partition was 88 % full. The Docker daemon was still running, but any new layer creation failed with no space left on device. Docker logs showed:

Error response from daemon: failed to create overlay mount: no space left on device

A quick docker system df revealed that the image layers were the largest contributor:

Images space usage: 4.2 GB
Containers space usage: 0.0 GB
Local volumes space usage: 0.0 GB
Build cache space usage: 0.0 GB

The docker system df command is a handy diagnostic tool that shows where Docker is eating space.


Immediate recovery steps

  1. Stop the Docker daemon to prevent further writes.

    sudo systemctl stop docker
    
  2. Clean up unused images and containers. Even though the build was failing, there were old images lingering from previous runs.

    sudo docker image prune -a -f
    sudo docker container prune -f
    
  3. Clear the build cache. Docker keeps a cache of intermediate layers that can be purged safely if you’re not using them for incremental builds.

    sudo docker builder prune -f
    
  4. Check the overlay filesystem for orphaned directories. Sometimes Docker leaves dangling directories if a build is interrupted.

    sudo find /var/lib/docker/overlay2 -type d -empty -delete
    
  5. Free up space in /var/lib/docker by moving the directory to a larger partition (if available) or by mounting a separate SD card for Docker storage.

    sudo systemctl stop docker
    sudo mv /var/lib/docker /mnt/extra/docker
    sudo ln -s /mnt/extra/docker /var/lib/docker
    sudo systemctl start docker
    

    This approach is useful if you have a second SD card or an external USB drive. The Pi’s SD card is limited, but a USB 3.0 drive can provide several terabytes of free space.


Long‑term fixes

1. Use a dedicated Docker storage location

Move Docker’s root directory to a separate partition or external drive. In /etc/docker/daemon.json add:

{
  "data-root": "/mnt/docker"
}

Then restart Docker. This keeps the OS partition cleaner and allows you to allocate more space to Docker.

2. Limit the size of the build cache

Docker’s build cache can grow unbounded. Set a maximum size with the --max-size flag when you run docker buildx:

docker buildx build --max-size 500m -t myapp:latest .

Alternatively, configure the builder:

docker buildx create --name mybuilder --driver docker-container --use
docker buildx inspect --bootstrap

Then add a --build-arg BUILDKIT_CACHE_MAX_SIZE=500m to your build script.

3. Use multi‑stage builds wisely

Multi‑stage builds help keep the final image small, but they still create intermediate layers. If you’re building a large language runtime, consider pulling the runtime as a base image instead of building it from source. For example:

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
CMD ["node", "dist/index.js"]

The builder stage is discarded after the final image is built, but the layers still consume space during the build. Using a smaller base image for the builder can reduce the temporary footprint.

4. Prune Docker automatically

Set up a cron job that runs docker system prune -af every night. Combine it with a check that the disk usage is above a threshold:

#!/usr/bin/env bash
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
if

See also