iotop shows which process is burning the disks

Wed, 15 Mar 2028 00:00:00 +0000

A disk that spikes with no obvious writer is hard to chase; iotop maps throughput straight back to the task doing the damage. System-wide tools show the symptom — high await, saturated utilization — but never the suspect: fifty processes share the disk and the average blames all of them equally. The question that ends the investigation is always per-process — who, how much, since when — and iotop answers exactly that, live, from kernel counters no application can hide from.

Keep it beside the system-wide view rather than instead of it. iostat or vmstat says the disk is the bottleneck; iotop names the process making it so. Together they compress a storage mystery into a two-command diagnosis: confirm the resource, identify the consumer.

Per-process counters

iotop reads per-process IO from the kernel, so live read and write rates map to a pid rather than a nameless load average. Run it with -o to show only processes that are actually doing IO. The default view splits reads from writes and shows swap-in percentage alongside throughput, which separates the genuinely disk-hungry from the merely present — a process with high DISK READ but zero writes is scanning, one with writes is producing, and the distinction decides whether you throttle, relocate, or kill.

Note what the counters actually measure: bytes passing through the IO layer per process, including buffered writes not yet on disk. A process can show huge write rates while the disk looks idle (page cache absorbing) or modest rates while the disk saturates (many small synchronous writes). Read iotop together with device utilization: process-side blame plus device-side saturation is the complete picture, and either half alone misleads.

See whatever sweeps the disk

Sort by bandwidth and the offender stands out, whether it is a log hoarder, an indexer, or a runaway backup. Follow the pid to the exact process (and often the file) before acting. The usual lineup: an unrotated log written at debug level, a reindex job scheduled against peak hours, a backup writing to the same spindle it reads from, a container layer unpack looping on a corrupt image. Each has a different fix — rotate, reschedule, separate, prune — and killing the process treats the symptom while the cause reschedules itself for tomorrow.

Resolve the pid to its full context before touching anything: command line, working directory, open files via lsof, parent process. A java process burning writes might be ten applications behind one PID namespace; the fix belongs to the right one. Attribution first, action second — every storage war story that ends in regret skipped the first step.

Watch before you act

iotop observes rather than changes anything, so you can investigate the culprit before deciding what to throttle, move, or kill. Pair it with strace or lsof on the pid to pin the file and the syscall. The combination is decisive: iotop says process 18412 writes 400 MB/s, lsof says the writes go to one growing log file, strace shows O_APPEND writes of 4 KB — diagnosis complete, and the fix (log level, rotation, async appender) is obvious instead of debated.

Record the baseline while healthy so the incident comparison means something. A five-minute iotop batch capture on a normal day — -b -n 60 to a file — gives you “normal for this host” in numbers: which processes usually lead, what rates are ordinary. Without it, every spike looks unprecedented; with it, most incidents classify in seconds as “the indexer, again, an hour early.”

When it breaks

The classic meta-failure is permission blindness: run without privileges and iotop shows question marks instead of process details, or hides other users’ processes entirely — and the actual offender, running as another user, stays invisible. Run with the necessary privileges (root or the right capabilities) from the start, or the tool exonerates exactly the processes you cannot see. An investigation that “found nothing” with unprivileged iotop found nothing about your own processes only.

The second failure is sampling illusion on bursty IO: a process doing violent one-second bursts every minute looks innocent in a 5-second average and guilty in a 1-second one, or vice versa. Vary the delay interval when the numbers and the symptoms disagree, and prefer accumulated totals (-a) over instantaneous rates for “who hurt the disk over the last hour” questions. The right aggregation answers the question asked; the default answers whichever question its interval assumes.

Check it quickly

Capture who does IO, with context, non-interactively:

iotop -b -n 5 -o -d 2 | head -n 30
pid=$(iotop -b -n 1 -o | awk 'NR>5 {print $1}' | head -n 1); ls -l /proc/$pid/exe; cat /proc/$pid/cmdline | tr '\0' ' '
iostat -x 1 3 | grep -E "Device|sd|nvme"

The batch capture ranks offenders without a terminal UI, the pid resolution names the top writer’s binary, and iostat confirms the device side is actually saturated.

Sources

iotop man page, Official iotop reference, checked 2026-06-01.
vmstat man page, documents system-wide CPU and IO-wait reporting, checked 2026-09-05.