I/O Monitoring

iostat, commonly provided by the sysstat package, reports CPU and block-device activity. Use repeated samples and application latency together: throughput or utilization alone does not establish whether storage is causing a user-visible problem.

Collecting Useful Samples

Run extended device statistics at one-second intervals:

$ iostat -xz 1

On common implementations, the first report contains averages since boot and later reports cover each interval. The -x option adds extended fields, while -z suppresses inactive devices. Allow several intervals to capture normal and problematic periods.

CheckWhat does the first iostat report commonly represent?

Only operations from the final second of the command. Activity averages since the system booted. A forecast of tomorrow's device utilization.

Reading CPU Fields

The CPU section commonly includes user (%user), system (%system), idle (%idle), I/O wait (%iowait), and virtual-machine steal (%steal) time. I/O wait is CPU idle time during which the system has an outstanding I/O request; it is not the percentage of a disk that is busy.

CheckWhat does %iowait describe?

The percentage of disk capacity already filled. CPU idle time while an I/O request is outstanding. The number of files waiting to be deleted.

Reading Device Fields

Field names vary by sysstat version, but useful concepts include:

• Read and write operations or data per second show workload rate.
• await reports average request latency, including queue and service time.
• Average queue-size fields show requests waiting or being serviced.
• %util reports the percentage of elapsed time during which the device had I/O in progress.

High %util can indicate saturation for a simple serial device, but it does not translate directly into performance capacity for parallel storage, arrays, or virtual devices. Compare latency with the device design, workload pattern, and service objective.

CheckWhich field is most directly associated with average I/O request latency?

Device name. await %idle

Correlating the Evidence

Map device names to mounts and backing devices before drawing conclusions:

$ lsblk -o NAME,TYPE,SIZE,FSTYPE,MOUNTPOINTS
$ findmnt

Then correlate iostat intervals with application response time, database or filesystem metrics, and process-level I/O. Device-mapper, RAID, containers, and network-backed storage can add layers that require their own tools.

CheckWhat should you do after seeing high %util on a device?

Assume every filesystem is out of free space. Delete files before identifying the mounted workload. Correlate latency and workload behavior with the storage design.

Summary

You can now use iostat as evidence in an I/O investigation.

1. Collect multiple extended-statistics intervals.
2. Distinguish CPU I/O wait from device busy time.
3. Interpret latency, queueing, throughput, and utilization together.
4. Map devices to workloads and verify application impact.