Linux Café

Dealing with Disk Space Emergencies

I’ve seen this go wrong when you least expect it - a small Linux server, whether in a homelab or for self-hosting, suddenly runs out of disk space. This can happen due to misconfigured log rotation, unexpected data growth, or simply underestimating storage needs. When it does, acting quickly and making informed decisions is crucial to prevent data loss and service downtime.

Identifying the Issue

The real trick is to identify the cause of the issue quickly. I usually start with df -h to get an overview of the disk space usage, or du -sh /path/to/directory to check the size of specific directories. For a more detailed analysis, tools like ncdu are invaluable - they provide an interactive way to browse through the directory tree and identify large files or directories. This is where people usually get burned, not realizing that a single large file or directory is consuming all their disk space.

Introduction to Slow Boot Issues

I’ve had my fair share of slow boot issues on Linux systems. Recently, I spent a significant amount of time troubleshooting a stubborn problem on one of my homelab machines. The system would take an inordinate amount of time to boot, and I couldn’t seem to pinpoint the cause. My first instinct was to use systemd-analyze blame to identify the culprit, but to my surprise, it didn’t provide any clear answers. I’ve seen this go wrong when the issue is more complex than a simple misconfigured service.

Introduction to systemd-resolved

I’ve been dealing with DNS lookup failures on my home network, and I’ve found that the systemd-resolved cache can be a major culprit. I’ve seen this go wrong when the cache becomes outdated or corrupted, leading to frustrating DNS lookup failures. In this article, I’ll share my experience with taming the systemd-resolved cache to fix these issues.

Understanding the systemd-resolved Cache

The real trick is to understand how the systemd-resolved cache works. It’s designed to improve DNS lookup performance by storing recent queries and their results. However, this cache can become outdated or corrupted, leading to DNS lookup failures. To get a better understanding of the cache, let’s take a look at the systemd-resolved configuration file, usually located at /etc/systemd/resolved.conf. You can view the file using the following command:

Introduction to Borg Backup

I’ve been using Borg Backup for years, and it’s been a reliable tool in my homelab setup. However, like any complex system, it’s not perfect, and I recently had to deal with a failed repository. This experience taught me a lot about the recovery process, and I’d like to share my findings with you.

Understanding Borg Backup Repositories

Before we dive into recovery, it’s crucial to understand how Borg Backup repositories work. A repository is essentially a storage location for your backups, and it can be hosted locally or remotely. Borg uses archives to store individual backups within the repository, and each archive contains a snapshot of your data at a particular point in time. I’ve seen this go wrong when the repository becomes inconsistent, so it’s essential to understand the basics.

Introduction to Disk Space Management

I’ve learned the hard way that managing disk space is crucial for a healthy and efficient home server. It’s easy to let disk space consumption get out of hand, especially with the amount of data we generate these days. I’ve seen this go wrong when I didn’t keep a close eye on my server’s disk usage - it was growing at an alarming rate. To get back on track, I’ve implemented a quarterly rsync snapshot and log rotation strategy. The real trick is to find a balance between keeping enough data for backups and not letting logs consume too much disk space.

Introduction to Background Tasks

I’ve been running Linux servers in my homelab for years, and one thing that’s always come up is the need to run background tasks. These can be anything from simple scripts to complex applications, and they need to keep running even when I log out. I used to rely on tools like screen and tmux, but lately, I’ve been using nohup and systemd to simplify things.

Introduction to Background Jobs

I’ve lost count of how many times I’ve needed to run a command or script in the background, freeing up my terminal for other tasks. This is especially useful for long-running jobs like backups, downloads, or compilations. However, managing these background jobs can get messy, particularly when dealing with multiple processes or disconnections from the terminal. That’s where nohup and systemd come in - two essential tools for keeping your background jobs in check.

Introduction to systemd-resolved

I’ve seen this go wrong when working with Linux systems - systemd-resolved taking over your DNS settings without warning. As a Linux user, you’re likely familiar with the resolv.conf file, which contains the DNS resolver configuration for your system. However, with the introduction of systemd-resolved, things have changed. systemd-resolved is a systemd service that provides DNS resolution and caching, and it can sometimes override the settings in your resolv.conf file. In this article, we’ll explore how to tame systemd-resolved and prevent it from overriding your resolv.conf settings.

Introduction to systemd-resolved

I’ve spent years working with Linux, and one component that’s given me trouble is systemd-resolved, the DNS resolver service that comes bundled with systemd. Recently, I noticed my DNS queries were leaking, and I was getting annoying resolver errors on my Linux desktop. This prompted me to dive deeper into systemd-resolved and figure out how to tame it. I’ve seen this go wrong when people don’t take the time to understand how it works, so let’s start with the basics.

Introduction to journalctl Filtering

I’ve been using journalctl for years, and I’ve learned that it can be overwhelming if you don’t know how to filter the noise. By default, journalctl displays a vast amount of information that may not be relevant to your current needs. In this article, I’ll share how I tame the noise in journalctl using custom filters and priorities.

Understanding journalctl Basics

Before we dive into filtering, let’s cover some journalctl basics. The command journalctl is used to query the systemd journal, which stores log messages from various system components, including systemd services, kernel messages, and other system logs. I usually start with a simple command to view the most recent journal logs: