How a Forgotten SSH Key Stopped My Nightly Off‑Site Borg Backup and the Quick Fix I Implemented

The night I lost an off‑site Borg backup

I run a 24/7 home‑lab server that backs up my home directories to a remote NAS with Borg. The backup is triggered by a systemd timer that fires every night at 02:00 UTC. For months it worked flawlessly, but one morning the timer ran, the job logged “ssh: connect to host …: Permission denied”, and the backup vanished into the night. The culprit? A forgotten SSH key.

[Read More]

Why ssh keeps spawning new processes on every connect and how ControlPersist solves it

Why SSH keeps spawning new processes on every connect

Every time I run ssh host, the client forks a new process that talks to the remote sshd.
On the server side, sshd spawns a child for each accepted connection.
The model is simple and works fine for a handful of sessions, but it starts to feel like a drain when you:

  • run a script that opens dozens of SSH connections in a row,
  • use CI/CD pipelines that hit a remote host repeatedly, or
  • have a home‑lab with many small services that need quick SSH access.

Each fork eats a few kilobytes of RAM and a handful of file descriptors.
If you hit the per‑user process limit (ulimit -u) or the system’s max_user_processes, you’ll see “Too many users” or “Connection refused” errors.
Even on a modest machine, the cumulative CPU cost of negotiating the SSH handshake over and over can add up.

[Read More]

I lost SSH access to my home server after accidentally setting /home/user to 777 – how I restored it with a live USB

I hit a wall yesterday when I accidentally set /home/user to 777 and lost SSH access.
The server was a 2026‑era Debian‑based homebox running systemd‑managed services.
SSH refuses to log in if any file under the user’s home directory is writable by others, so the key‑based login failed immediately.
The only way to fix it was to boot from a live USB, mount the root filesystem, and restore the correct ownership and permissions.

[Read More]

Using SSH-Agent and Key Forwarding to Simplify Jump Host Hops

Introduction to SSH-Agent and Key Forwarding

I’ve been using SSH for years, and one thing that’s always been a bit of a hassle is managing SSH keys and connections when working with multiple servers or jump hosts. That’s where SSH-Agent and key forwarding come in - they can simplify your SSH workflow and make it easier to access remote systems. In this article, I’ll cover how to use these features to improve your SSH experience.

[Read More]