On a small home‑lab server or a single‑user laptop, the root filesystem is often the first place that runs out of space. I ran into this on a 2025‑era Raspberry Pi running Ubuntu 24.04 LTS: the systemd journal kept growing until the root partition hit 95 % and the machine refused to boot. The fix was surprisingly simple once you understand how systemd‑journald works and how to steer its output away from the root filesystem.

Why /var/log is a problem on a small system

/var/log is the default location for both the traditional syslog files and the binary systemd journal. On a machine with a 32 GB SD card or a 64 GB NVMe, the journal can balloon quickly if you have services that log a lot of debug output. Unlike logrotate, systemd‑journald does not rotate files by default; it keeps appending to a single binary database until the configured limits are hit. When those limits are not set, the journal can grow until the partition is full.

The symptoms are easy to spot: df -h / shows 100 % usage, journalctl --disk-usage reports a large number, and boot logs are truncated. The root cause is that the journal is stored in /var/log/journal, which lives on the root filesystem unless you explicitly move it.

Understanding systemd‑journald storage options

systemd‑journald offers two storage modes:

volatile – stores the journal in memory (/run/log/journal). It is wiped on reboot. Good for containers or systems where persistence is not required.
persistent – stores the journal on disk under /var/log/journal. This is the default on most desktop distributions.

You can view the current setting with:

systemctl show --property=Storage systemd-journald.service

If you see Storage=persistent, the journal will consume disk space. To change it, edit /etc/systemd/journald.conf:

[Journal]
Storage=volatile

Then reload:

sudo systemctl restart systemd-journald

This immediately stops the journal from writing to disk, but you lose historical logs. For a production server, you usually want a persistent journal on a separate partition.

Moving the journal to a dedicated partition

The cleanest solution is to create a small ext4 or btrfs partition dedicated to logs. On a 64 GB SSD, a 4 GB partition is more than enough for most workloads.

# Create a 4 GB partition on /dev/nvme0n1
sudo fdisk /dev/nvme0n1
# (use n, p, 1, +4G, w)

# Format it
sudo mkfs.ext4 /dev/nvme0n1p1

# Mount it temporarily
sudo mkdir /mnt/logs
sudo mount /dev/nvme0n1p1 /mnt/logs

# Copy existing logs
sudo rsync -a /var/log/journal/ /mnt/logs/

# Rename old directory
sudo mv /var/log/journal /var/log/journal.bak

# Create a bind mount
sudo mkdir /var/log/journal
sudo mount --bind /mnt/logs /var/log/journal

# Make it permanent
echo '/dev/nvme0n1p1 /var/log/journal ext4 defaults 0 0' | sudo tee -a /etc/fstab

Now /var/log/journal points to the new partition, and the root filesystem is freed. If you prefer a separate device, you can also use a USB stick or a network share, but a local partition gives the best performance.

Fine‑tuning journal retention with systemd‑config

Even with a dedicated partition, you still want to limit how much the journal can grow. Edit /etc/systemd/journald.conf and set:

[Journal]
SystemMaxUse=500M
SystemKeepFree=200M
SystemMaxFileSize=50M
SystemMaxFiles=20

SystemMaxUse caps the total size of all journal files.
SystemKeepFree reserves free space for the system.
SystemMaxFileSize limits the size of each individual file.
SystemMaxFiles limits the number of files.

After editing, restart the service:

sudo systemctl restart systemd-journald

You can verify the limits with:

journalctl --disk-usage

The trade‑off is between log retention and disk usage. If you need to audit old events, keep SystemMaxUse higher; if you only care about recent activity, lower it.

Using logrotate for legacy syslog files

systemd‑journald does not rotate the traditional /var/log/*.log files. If you have services that still write to syslog (e.g., rsyslog or syslog-ng), you should configure logrotate. On Ubuntu, the default /etc/logrotate.d/syslog rotates daily and keeps seven weeks of logs:

/var/log/syslog
{
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 syslog adm
}

If you moved /var/log/journal to a separate partition, you might also move /var/log/syslog there to keep all logs together:

sudo mv /var/log/syslog /mnt/logs/syslog
sudo ln -s /mnt/logs/syslog /var/log/syslog

Now both journal and syslog share the same storage pool.

Monitoring disk usage and automating cleanup

A small cron job can keep an eye on the journal size and alert you before it fills the root filesystem:

#!/usr/bin/env bash
THRESHOLD=90
USAGE=$(df / | awk 'NR==2{print $5}' | tr -d '%')
if [ "$USAGE" -ge "$THRESHOLD" ]; then
    echo "Disk usage at ${USAGE}% – journal size: $(journalctl --disk-usage)" | mail -s "Disk alert" [email protected]
fi

Save this as /usr/local/bin/check-journal.sh, make it executable, and add it to cron:

crontab -e
0 * * * * /usr/local/bin/check-journal.sh

For automated cleanup, you can rely on the journal’s own retention settings, but if you need a manual purge, run:

sudo journalctl --vacuum-size=200M

sudo journalctl --vacuum-time=7d

The first command keeps only the most recent 200 MB; the second keeps only logs from the last 7 days.

Security implications of journal retention

The systemd journal is a powerful audit tool. Trimming it too aggressively can make forensic investigations difficult. On the other hand, keeping logs for too long can expose sensitive data (e.g., authentication attempts, configuration changes). Find a balance that suits your threat model and compliance needs.

TAGS: linux journald logs raspberry-pi systemd

linux open-source

How I Stopped /var/log from Flooding the Root Filesystem with Systemd Journal Rotations