Diagnosing the cause of high server load on a Linux server involves examining various aspects of system performance, including CPU, memory, disk I/O, and network usage. Here’s a structured approach using different tools and commands to identify the underlying cause of the high load:
1. Check High Server Load
uptime and top
- uptime: Quickly shows load averages.
uptimeOutput:
15:24:19 up 10 days, 2:35, 1 user, load average: 0.72, 1.01, 1.29- top: Provides a real-time view of system performance.
top2. Analyze CPU Usage
top and htop
- top: Check CPU usage.
top -o %CPUPress Shift + P to sort by CPU usage.
- htop: An enhanced version of top.
sudo apt-get install htop (On Ubuntu Machine)
sudo yum install htop htop (On Centos Machine)Allow for easier navigation and better visualization.
mpstat
- mpstat: Part of the sysstat package, useful for checking CPU usage by individual cores.
mpstat -P ALL 13. Check Memory Usage
free and vmstat
- free: Shows memory usage.
free -h- vmstat: Provides detailed memory, CPU, and I/O statistics.
vmstat 14. Inspect Disk I/O
iostat , iotop , and dstat
- iostat: Part of sysstat , useful for checking I/O statistics.
iostat -dx 1- iotop: Monitors I/O usage by processes.
sudo iotop- dstat: A versatile tool for various system statistics, including disk I/O.
dstat -d --disk-util --disk-tps5. Examine Network Usage
ifstat , iftop , and netstat
- ifstat: Displays network interface statistics.
sudo apt-get install ifstat (On Ubuntu Machine)
sudo yum install ifstat (On Centos Machine)
ifstat 1- iftop: Monitors bandwidth usage by host.
sudo apt-get install iftop (On Ubuntu Machine)
sudo yum install iftop (On Centos Machine)
sudo iftop- netstat: Provides detailed network statistics.
netstat -at6. Check for Heavy Processes
ps and pgrep
- ps: Lists running processes. Sort by memory or CPU usage.
ps aux --sort=-%mem | head -n 10 # Sort by memory
ps aux --sort=-%cpu | head -n 10 # Sort by CPU- pgrep: Finds processes based on name and other attributes.
pgrep -l high_load_process_name7. Look for System Logs
dmesg and /var/log
- dmesg: Shows kernel and driver messages.
dmesg | tail- /var/log/syslog or /var/log/messages: Check for system errors or warnings.
tail -f /var/log/syslog8. Analyze Running Services
systemctl and service
- systemctl: Check the status of running services.
systemctl list-units --type=service --state=running- service: Legacy command for managing services.
service --status-all9. Check for Zombie Processes
top and ps
- top: Look for processes with status Z (zombie)
- ps: List zombie processes.
ps aux | grep 'Z'10. Monitor System with sar
sar can collect, report, and save system activity information.
- Install sysstat:
sudo apt-get install sysstat # For Debian/Ubuntu
sudo yum install sysstat # For CentOS/RHEL- Use sar to review CPU, memory, and I/O stats.
sar -u 1 5 # CPU usage
sar -r 1 5 # Memory usage
sar -d 1 5 # Disk I/OSummary of Commands
Here’s a quick reference for the commands mentioned:
- System Load:
uptime top- CPU Usage:
top -o %CPU htop mpstat -P ALL 1- Memory Usage:
free -h vmstat 1- Disk I/O:
iostat -dx 1 sudo iotop dstat -d --disk-util --disk-tps- Network Usage:
ifstat 1 sudo iftop netstat -at- Processes:
ps aux --sort=-%mem | head -n 10 ps aux --sort=-%cpu | head -n 10- Logs:
dmesg | tail tail -f /var/log/syslog- Services:
systemctl list-units --type=service --state=running service --status-all- Zombie Processes:
ps aux | grep 'Z'- sar Monitoring:
sar -u 1 5 sar -r 1 5 sar -d 1 5Using these tools and commands, you should be able to pinpoint the cause of high load on your Linux server.
