Linux Café

Auditing Stale SSH Keys with ssh-keygen -l and Protecting with fail2ban

SSH keys that sit unused for months can become a silent backdoor if a private key ever gets compromised. On a production server the safest way to keep the key list lean is to audit it regularly and block any brute‑force attempts that surface while you’re cleaning up.

1. List all authorized keys with fingerprints

The authorized_keys file lives in each user’s ~/.ssh/. To see which keys are actually present, run:

Automating Let’s Encrypt renewal on a single‑node web server without downtime

Let’s Encrypt certificates expire every 90 days. On a single‑node web server the usual approach is to run certbot renew from cron and reload the web service. The trick is to avoid any traffic loss while the new cert is in place. Below is a minimal, production‑ready recipe that works with Apache, Nginx, or any systemd‑managed web server.

1. Install Certbot and the web‑root plugin

# Debian/Ubuntu
sudo apt-get install certbot python3-certbot-nginx   # or python3-certbot-apache

# RHEL/CentOS
sudo dnf install certbot python3-certbot-nginx

The nginx or apache plugin automatically configures the web‑root challenge and reloads the server after a successful renewal.

Disk exhaustion is the most common cause of a sudden “500 Internal Server Error” on a small VPS.
When the filesystem that holds /var/log/apache2 is full, Apache can’t rotate or write new logs, and the web server silently stops accepting requests.
The fix is usually a quick cleanup, but doing it manually every time is error‑prone and can leave you with the same problem again.
Below is a practical, repeatable script that:

Disk‑full crashes are the most common reason a VPS goes from “alive” to “dead” overnight.
A log file or a database that blows past the allotted space will make the kernel refuse any further writes, and every process that needs to touch the disk will start failing. If you’re running a website, mail server, or database, that’s a service outage you can’t afford.

The silver lining? A handful of LVM tricks and a tiny monitoring script can get you back up in minutes and keep future incidents at bay.
Below I walk through a real‑world scenario on a Debian‑based VPS, but the ideas carry over to any distro that uses LVM and systemd.

Rootless Podman Volumes: Keeping Your Data Safe Across Upgrades

Rootless Podman lets you run containers without ever touching sudo.
That’s great, but it also means every volume lives in your home directory.
When you bump Podman or your distro, the storage layout can change and all your data can get wiped.
Below is a practical, coffee‑talk style walk‑through that shows how to sidestep this trap.

Why Rootless Volumes Are Different

When you run

ProxyJump timeouts when a VPN drops your IP address

When you hop through a bastion with ssh -J, the jump host keeps a TCP tunnel open to the target.
If you’re on a VPN that can change your public IP or even tear the tunnel down, that tunnel can stall or die.
The result? A hanging SSH that eventually spits out “Connection timed out” or “Connection reset by peer”.
For homelab users who rely on VPNs for remote access, this is the kind of annoyance that keeps you awake at night.

Keep a VPN alive when the home router flips between DHCP and static

I’ve been on a laptop that plugs into a home router that sometimes hands out a DHCP lease and other times uses a static IP you set in the admin UI. The VPN client (OpenVPN 3‑CLI on Ubuntu 24.04) would drop the tunnel as soon as the interface lost its address, forcing me to reconnect manually. Below is a minimal, reproducible setup that keeps the tunnel up regardless of the router’s mode.

Running a production MySQL instance on a small home‑lab server is a common scenario for many of us. The database is the heart of the service, but it also means that any backup strategy must preserve consistency while keeping the application online. Restic is a popular, fast, deduplicating backup tool that works well with block‑level snapshots and remote backends. In this post I’ll walk through a practical, repeatable method to test incremental Restic backups against a live MySQL server, covering the trade‑offs, commands, and pitfalls you’ll encounter.

The 2‑second DNS lookup lag on a Pi – a hands‑on fix

I was running a Pi‑4‑based home‑lab that hosts a few services: a personal Git server, a small reverse‑proxy stack, and a few IoT sensors that ping the Pi every few seconds. Every time I pinged a domain that wasn’t in the local cache, I noticed a ~2 s pause before the first packet left the network interface. On a desktop that latency was negligible, but on the Pi it was a noticeable hiccup that made the sensor logs look a little jittery.

The 30‑second Nginx start‑up mystery on an 8‑core ARM box

I had a clean 8‑core ARM server (Cortex‑A72, 4 GB RAM) running Debian 12. Nginx was the only service that took longer than a couple of seconds to come up. The log said “started” after 30 s, but the system was idle otherwise. I wanted to know why a stateless web server was so slow to boot on a modern CPU.