Locating Disk I/O Bottlenecks for Containers

In some situations, the disk I/O subsystem may bottleneck a Virtuozzo server and reduce the performance of Containers or the server itself. Such I/O bottlenecks are often caused by disk-intensive processes running in Containers, for example, by transferring huge amounts of data to or from a Container.

To pinpoint Containers generating the highest disk I/O load, follow these steps:

1. On the problem server, run the pstat utility to display I/O statistics for all running Containers. You may want to use the following I/O options with the utility: id, cpu_iowait, cpu_iowait_acc, io, iops, swapin. For example:

   # pstat -o id,cpu_iowait,cpu_iowait_acc,io,iops,swapin

   ...

   CTID CPU IOWAIT ACC IOWAIT IO IOPS SWAPIN

   1 0.00% 0.00% 0.0/---KB/s 0.0/--/s 0.0/s

   101 0.00% 0.00% 0.0/---KB/s 0.0/--/s 0.0/s

   111 25.0% 28.0% 23/---MB/s 419/--/s 0.0/s

2. In the command output, find the Container with the highest values. Most probably, this is the Container that generates the highest I/O load on the server. In the example above, this is Container 111.
3. Once you have found the problem Container, run the vziotop -o -E <CT_ID> command to show all threads running in this Container and generating I/O load. To sort the output by I/O load, use the left and right arrow keys to select the IO column. For example:

   # vziotop -o -E 111

   Total DISK READ : 4.77 M/s | Total DISK WRITE : 270.00 K/s

   Actual DISK READ: 4.77 M/s | Actual DISK WRITE: 354.63 K/s

   CTID TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND

   111 1026959 be/4 root 932.72 K/s 250.00 K/s 0.00% 42.84% {output skipped];

   111 1026960 be/4 root 202.20 K/s 20.00 K/s 0.00% 11.53% {output skipped];

   The command output shows that thread 1026959 produces the highest I/O load and might be responsible for the server performance degradation.

For more information on the vziotop utility and its options, see the Virtuozzo 6 Command Line Reference Guide.