If a Linux server isn't performing well, it is often related to the storage channel. A few decades ago, this was...
relatively easy to analyze: the server had a RAID array, there were partitions on top of the RAID array and an Ext2 file system ran on top of the partitions. In today's data centers, however, the storage channel is not so easy to analyze.
Many Linux servers in modern data centers run on top of a VMware hypervisor that may connect to different types of storage area network (SAN) systems. That means there are a lot more factors to consider for Linux storage optimization.
Common sense says that when you use Linux on top of a hypervisor, you don't have to do anything regarding storage optimization, but that isn't the case in many situations. Storage performance depends on many factors, and one of those factors, the Linux I/O scheduler, has a significant effect when tuned correctly.
Understand different Linux I/O scheduler types
An I/O scheduler is the kernel process that determines how I/O requests are ordered. There are different types of schedulers: deadline, Completely Fair Queuing and noop (no operation). On older kernels, there is the anticipatory scheduler, as well.
The default Linux I/O scheduler for most systems is Completely Fair Queuing. With this scheduler, the Linux kernel tries to evenly distribute between read and write requests before sending them deeper into the storage channel. Most hypervisors and corporate SAN products do the same thing, so this scheduler is more likely to give a small degradation for specific loads than an improvement. It is the safest option to select, though, which is why all distributions use it as a default setting.
Many IT pros believe the noop scheduler offers the best possible performance when you use a smart underlying storage channel. With this scheduler, the Linux kernel directly transfers write and read requests to the storage channel, which reorders them. In most cases where a hypervisor, SSDs or SAN filer is used, noop offers the best possible performance. However, this might not always be the case -- particularly with heavily write-oriented loads -- and it may help the underlying storage channel to use the deadline scheduler.
The deadline I/O scheduler optimizes write requests by reordering them in the most efficient way, which eases the performance load on the underlying hypervisor layer. If your server is writing a lot, the deadline scheduler is worth a try.
Last, you may encounter the anticipatory scheduler. This scheduler was used in older Linux kernels and isn't very common anymore. On these older kernels, this scheduler optimizes read requests by performing a read ahead when allocating file storage blocks.
Set the Linux I/O scheduler
Admins can set the Linux I/O scheduler on specific disks or for the entire server. To set it for the entire server, modify the grub configuration file/etc/default/grub. In this file, locate the line that starts with Linux. In some distributions, Linux may be followed by a number. This line has all kernel boot arguments. On this line add elevator=setting, where "setting" needs to be changed by the I/O scheduler that you would like to use. After you change the GRUB configuration file, run grub2-mkconfig -o /boot/grub2/grub.cfg to write the new settings to your system, and then reboot the system.
While changing the Linux I/O scheduler for the entire system can work on some specific workloads, consider per-disk I/O scheduler settings as an alternative. Consider running tests with these settings if your server has different storage loads where the different load types are writing to different devices.
Every disk device has an interface file with the name /sys/block/device/queue/scheduler. You can echo the required scheduler setting to this file to make it effective immediately, as in echo deadline > /sys/block/sda/queue/scheduler. Linux does not provide a standard configuration file where this setting can be configured persistently, so you'll need to integrate it somewhere in your system startup scripts to automate it.
Identify possible problems for Linux VMs on Hyper-V
Run Linux on a VM
Schedule, automation and backup tools for Linux