The Linux kernel uses a system of active and inactive memory to manage a server's memory resources. If your Linux server is operating with a heavy load of these resources, there are a couple of possible causes: memory that's insufficient to load applications or to cache all files that should be in cache, or relying too heavily on Linux memory swap.
You can optimize a server based on Linux memory usage best practices, such as moving inactive memory to swap. Still, each individual load problem requires a Linux memory analysis before you can solve it. Let's look at a few scenarios and solutions.
Insufficient memory to load applications
A server that doesn't have enough memory to load applications suffers from a general shortage of memory. The server will show a relatively limited amount of memory in buffers and cache, and relatively heavy use of swap space. One easy fix is to add more memory. But optimizing Linux memory swap also can improve performance. Make sure inactive memory pages are swapped out sooner to make room for memory pages that are necessary.
To tell the server to start swapping sooner, use this command: echo 80 > /proc/sys/vm/swappiness. Also write the parameter to the /etc/systcl.conf file, using the following syntax: vm.swappiness = 80. If this doesn't give you more memory to load applications, then resort to adding more physical memory to the server.
Insufficient memory to cache files
If you suspect there is insufficient memory to cache files, follow this simple calculation: If the available cache and buffer memory is less than 30% of total RAM, your server might suffer from insufficient memory. Next, check the contents of /proc/meminfo. If the amount of inactive (file) memory is significantly higher than the amount of cache, your server doesn't have enough available memory to cache files. Before you can free up more cache, pinpoint what's causing the shortage.
If the programs that Linux loads are requesting too much memory, you don't have many options other than to add more physical RAM. However, other parts of the system using memory could also cause the cache shortage. For instance, it might take too long to commit changed files to disk. If this is the case, you will see a relatively high wa (waiting for I/O) parameter in the top program, and in /proc/meminfo you will see a relatively high value for the Dirty parameter. If this is the case, fixing disk performance is the key to better memory performance.
Memory swap is used too heavily
If swap is used too heavily, the vmstat monitoring tool will show high si and so (swap in and swap out) activity; the swap space will be mostly filled, and the server will have a low amount of free memory.
To optimize Linux memory conditions, first investigate the root cause of heavy swapping. In general, if only inactive (anonymous) memory is in swap, it shouldn't be under stress. In /proc/meminfo, see if the amount of swap used is much higher than the amount of inactive (anonymous) memory. If so, active memory pages are being swapped as well, even though the Linux kernel needs to access active memory pages frequently. The system needs more physical memory.
If swap is under stress and a large amount of cache is allocated, you can ease the memory load on the server by increasing the swappiness parameter, as described earlier. Higher swappiness value makes the kernel drop memory pages to disk sooner, relieving the stress from cache and making more memory available for active (anonymous) memory.
A higher swappiness parameter actually reduces disk thrashing caused by pages moving in and out of swap too frequently: Increasing swappiness just ensures that pages are moved to swap that should be there, calming down the swap activity on your server. To make this work, you might have to add some more swap to your server as well.
About the author:
Sander van Vugt is an independent trainer and consultant based in the Netherlands. He is an expert in Linux high availability, virtualization and performance. He has authored many books on Linux topics, including Beginning the Linux Command Line, Beginning Ubuntu LTS Server Administration and Pro Ubuntu Server Administration.