How I Prevented a Rootless Podman Container From Filling My Disk With Logs

The Problem

Running a rootless Podman container on a home‑lab server is great for isolation, but the logs it spits out can quietly eat up disk space. I once had a simple nginx container that, after a few days, filled a 20 GB SSD with log files. The container was running as my normal user, so I couldn’t just rm -rf /var/lib/containers/... without risking a permission error. The real issue was that Podman, by default, writes container stdout/stderr to files under /var/lib/containers/storage/overlay-containers/.../files/var/log/ and those files never rotate.

Rootless Podman and Log Storage

Rootless containers use the user‑namespace overlayfs layout. All container data lives in $XDG_RUNTIME_DIR/containers or $HOME/.local/share/containers. When a container writes to /var/log/nginx/access.log, Podman copies that into the overlay and keeps it there. Because the container runs as a non‑root user, the host’s systemd journal is not automatically used unless you explicitly tell Podman to forward logs. Without any rotation, a busy web server can generate gigabytes of log data in a week.

Strategies to Keep Disk in Check

Below are the tactics I use on my own machines. Pick the ones that fit your workflow; you can combine them for maximum safety.

1. Rely on Journald

The simplest way to avoid log files on disk is to let the container write to the host’s journal. Podman supports the journald log driver:

podman run --log-driver=journald \
           --log-opt=label=container=myapp \
           --log-opt=tag={{.ImageName}} \
           -d nginx:alpine

All output goes into journalctl under the specified label. You can then prune old entries with:

journalctl --vacuum-size=500M

or keep only the last 7 days:

journalctl --vacuum-time=7d

Because journald stores logs in a compressed, rotating format, you won’t run out of space unless you keep the retention window too long.

2. Use Podman Log Options

Podman 4.0+ added max-size and max-file options for the json-file driver, mirroring Docker’s behaviour:

podman run --log-driver=json-file \
           --log-opt=max-size=10m \
           --log-opt=max-file=5 \
           -d myapp

This keeps each log file under 10 MiB and keeps only five rotated files. The files live in the container’s overlay, so they’re still part of the rootless storage, but the size is capped.

3. Systemd User Units

When you run a container via a systemd user unit, you can set StandardOutput=journal and StandardError=journal so that systemd captures the logs. Example unit (~/.config/systemd/user/nginx.service):

[Unit]
Description=Rootless Nginx
After=network-online.target

[Service]
ExecStart=/usr/bin/podman run --rm -p 80:80 nginx:alpine
StandardOutput=journal
StandardError=journal
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=default.target

Enable and start it:

systemctl --user enable --now nginx.service

Now the logs are in the user’s journal, which you can prune with the same journalctl --vacuum-* commands.

4. Logrotate Outside the Container

If you prefer file‑based logs, set up a logrotate rule that points to the container’s log directory. Create /etc/logrotate.d/podman-nginx:

/home/youruser/.local/share/containers/storage/overlay-containers/*/files/var/log/nginx/*.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    create 0640 youruser youruser
}

logrotate will run daily and keep a week of compressed logs. Adjust the path if you use a different storage location.

5. tmpfs for /var/log in the Container

For containers that generate a lot of short‑lived logs, mount /var/log as a tmpfs. This keeps logs in RAM and they vanish when the container stops:

podman run -d \
  --tmpfs /var/log:rw,size=100m \
  nginx:alpine

The trade‑off is that logs are lost on reboot, but for stateless services it’s often acceptable.

Trade‑offs and Security

Journald vs. Files: Journald compresses and rotates automatically, but you lose the ability to stream logs to external systems without extra tooling. File logs are portable but need explicit rotation.
Podman Log Options: The max-size/max-file flags are easy but still keep logs on disk. If the container writes a burst of data, you might hit the size limit before rotation kicks in.
Systemd Units: Capturing logs in the user journal keeps them separate from root logs, which is good for isolation. However, the user’s journal can grow if you forget to vacuum.
tmpfs: Fast and clean, but logs are lost on shutdown. Also, RAM usage can spike if the container writes a lot of data.

From a security standpoint, keeping logs in the journal avoids exposing raw log files to the host filesystem, reducing the attack surface. If you need to audit logs, the journal’s structured format is easier to parse with journalctl.

Practical Commands and Examples

# Start nginx with journald logging
podman run --name nginx \
           --log-driver=journald \
           -d nginx:alpine

# Check journal entries
journalctl -u nginx --since "1 hour ago"

# Prune old journal entries
journalctl --vacuum-time=30d

# Run container with log rotation
podman run --name myapp \
           --log-driver=json-file \
           --log-opt=max-size=5m

linux podman-logs-rootless-journald-linux