Key takeaways
- On a NOC shift you are asked five questions on repeat: Is it up? Is it slow? Is the disk full? What changed? Where are the logs? Learn the commands that answer those, in that order.
- Prefer modern tools (
ss,ip,journalctl,nftables) but know the legacy ones (netstat,ifconfig) because the servers you inherit will be old. - Never run a command you cannot explain in a ticket note. Seniors judge you on the note, not the fix.
- Practise on a VM you are allowed to break. Our cloud engineer roadmap has the lab plan.
"Is it up?"
uptime # load averages, how long since reboot
systemctl status nginx # is the service running, last log lines
systemctl list-units --failed
ss -tulpn # which ports are listening, by which process
curl -I https://site.example # HTTP headers; -sv for the TLS handshake
ping -c 3 8.8.8.8 ; ping -c 3 google.com # network vs DNS
dig +short site.example # DNS answer (install bind-utils / dnsutils)
"Is it slow?"
top # or htop; press 1 for per-core, M to sort by memory, P by CPU
vmstat 1 5 # r (run queue), wa (I/O wait), si/so (swapping)
iostat -xz 1 3 # disk busy %, await (install sysstat)
free -h # memory; "available" is the number that matters
ps aux --sort=-%cpu | head # top CPU hogs
ps aux --sort=-%mem | head
nproc ; lscpu # how many cores you actually have
Rule of thumb: load average above the core count and high wa = disk; high us = CPU; si/so non-zero = out of RAM.
"Is the disk full?"
df -h # filesystem usage; watch /var and /
df -i # inodes — "disk full" with space free means inodes
du -xsh /var/* | sort -h # what is eating it
du -xsh /var/log/* | sort -h | tail
find /var/log -type f -size +500M
lsof +L1 # deleted-but-open files still holding space
journalctl --disk-usage ; journalctl --vacuum-size=500M
"What changed?"
last -n 20 # logins
journalctl -u nginx --since "1 hour ago"
journalctl -p err -b # errors since boot
grep -i "oom\|killed process" /var/log/syslog /var/log/messages
ls -lt /etc | head # recently modified configs
rpm -Va | head / debsums -c # modified package files
history ; cat ~/.bash_history # what the last person ran
"Where are the logs?"
journalctl -f # follow everything
tail -f /var/log/nginx/error.log
less +F /var/log/syslog # follow inside less; Ctrl-C to search, F to resume
grep -n "500" /var/log/nginx/access.log | tail
zgrep "pattern" /var/log/*.gz
Networking you will be asked about
ip a ; ip r ; ip -s link # addresses, routes, interface errors
ss -s # socket summary
traceroute / mtr host # where the path breaks
nc -zv host 443 # is the port reachable
tcpdump -i eth0 port 443 -nn -c 50 # capture (needs root)
nmcli device status
Files, users, services
chmod / chown / ls -l # permissions — know rwx and octal
sudo -l # what you are allowed to run
useradd -m / passwd / usermod -aG
systemctl enable --now svc ; systemctl daemon-reload
crontab -l ; ls /etc/cron.d
rsync -av --progress src/ dst/
tar -czf backup.tgz /etc # and tar -xzf to restore
The one-liners seniors keep
# top 10 IPs hitting the web server right now
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head
# which process holds a port
ss -tulpn | grep :3306
# watch a command every 2 seconds
watch -n 2 'df -h /var'
# disk I/O per process (install iotop)
iotop -oPa
# certificate expiry
echo | openssl s_client -connect site.example:443 2>/dev/null | openssl x509 -noout -dates
Writing the ticket note
Symptom: nginx 502 on www since 10:42 IST
Checked: systemctl status php-fpm → failed; journalctl shows OOM kill 10:41
Cause: memory exhausted by runaway cron job (reports.php)
Fix: killed job, restarted php-fpm 10:48; service verified with curl -I
Follow-up: raise RAM alert threshold, review cron job (ticket #1234)
A note like that, every time, is how you get out of L1.
Want to practise against real alerts? We run NOC services and train L1/L2 engineers — see NOC & SOC setup.