Linux Server Troubleshooting

When a server misbehaves, stay calm and work through these dimensions step by step.

High system load

uptime                    # load averages
top                       # real-time processes, sort by CPU/memory
htop                      # friendlier top
ps aux --sort=-%cpu | head    # top CPU consumers
ps aux --sort=-%mem | head    # top memory consumers

The three load average numbers should be near your core count. If they exceed it several-fold, the box is overloaded.

Out of memory / OOM

free -h                   # memory overview
dmesg | grep -i oom        # see which process was killed

After identifying the culprit, use kill <pid> or optimize the application’s memory usage.

Disk space

df -h                     # partition usage
du -sh /var/* | sort -h   # find big directories
du -h --max-depth=2 /var/log | sort -h | tail
journalctl --disk-usage   # journal log usage

Log cleanup:

sudo journalctl --vacuum-size=200M

Ports in use

ss -tlnp                  # listening ports and processes
ss -tlnp | grep :80
lsof -i :8080             # who owns port 8080

Network issues

ping -c 4 example.com     # connectivity
curl -I http://localhost  # local service response
traceroute example.com    # route tracing
ss -s                     # connection stats

DNS resolution checks:

dig example.com +short
nslookup example.com

A service won’t start

sudo systemctl status myservice        # status and recent logs
sudo journalctl -u myservice -n 50     # last 50 log lines
systemctl list-unit-files | grep myservice   # enabled at boot?

Abnormal processes

pgrep -a nginx            # find by name
kill -9 <pid>             # force kill (last resort)
ps -ef --forest           # process tree, find orphans

A general debugging order

1. System resources first: uptime, free -h, df -h
2. Then service state: systemctl status, logs
3. Finally network: listening ports, DNS, connectivity

Follow the order and most problems can be pinned down within ten minutes.