Resource allocation with Linux cgroups helps optimize performance

The Linux cgroups feature lets you make a profile to manage resources to keep your system under control.

Among the latest features for performance optimization with Linux, control groups allow you to create and allocate...

groups of resources to specific services. Control groups ensure a fixed percentage of resources on your server are available for those services.

Start with a recent Linux kernel that supports control groups (cgroups). You can use the latest versions of enterprise Linux distributions like Red Hat Enterprise Linux (RHEL), SUSE Linux Enterprise Server (SLES) or any other Linux distribution on kernel version 2.6.24 or later. Here we're using RHEL 6.2.

To configure your cgroups, make sure the libcgroup software package is installed. Once installation is complete, start the cgconfig and cgred services in the runlevels of your server using chkconfig cgconfig on and chkconfig cgred on. This creates a directory /cgroup with a couple of subdirectories — called controllers — which refer to the system resources that you can limit using cgroups. The ones we need are:

  • blkio:    Use this to limit the amount of I/O that can be handeld
  • cpu: Limit CPU cycles.
  • memory: Limit the amount of memory granted to processes.

Now let's assume that you're running Oracle on your server and you want it to have access to at least 75% of available memory and CPU cycles. First, create a cgroup that defines access to CPU and memory resources. The following command would create and name the "oracle" cgroup: cgcreate -g cpu,memory oracle. Defining the cgroup this way gives you a subdirectory, oracle, in the /cgroups/cpu and /cgroups/memory directories. In this subdirectory, different parameters are available to specify the resources you want to make available to the cgroup.

Some of the tunable parameters in your cgroup's subdirectory:

[[email protected] ~]# cd /cgroup/cpu/oracle/
[[email protected] oracle]# ls
cgroup.procs              cpu.rt_period_us       cpu.stat
cpu.cfs_period_us     cpu.rt_runtime_us    notify_on_release
cpu.cfs_quota_us      cpu.shares                   tasks

To specify the percentage of CPU resources available for the oracle cgroup, use the cpu.shares parameter. This relative parameter only makes sense if everything is in cgroups. So if you'll give it value 80 and you give the cgroup other — containing all other processes — value 20, the oracle cgroup gets 80% of available CPU resources. To set this parameter, use the cgset command: cgset -r cpu.shares=80 oracle.

Once you set the amount of CPU shares for this cgroup, you can start adding processes. The best way to do this is to start the process you want to put in the cgroup as an argument to the cgexec command. In this example you'd run cgexec -g cpu:/oracle /path/to/oracle. Now the oracle process and all its child processes will be visible in the /cgroups/cpu/oracle/tasks file. You have successfully assigned Oracle to its specific cgroup.

You've now read how to manually create cgroups, make resources available to the cgroup and put a process in it. You won't want to leave it at that because a system restart all lose those settings. We'll use the cgconfig service and the cgred service to prevent this. The cgconfig service reads a configuration file — we'll call ours /etc/cgconfig.conf — in which the cgroups and their assigned resources are defined. Here's what it would look like with the oracle example we've used so far.

To create the cgconfig.conf file:

group oracle {

                cpu {



                memory {



Next, create cgrules.conf — a file that specifies the processes that will be put in a specific cgroup automatically. This file is read when the cgred service starts. In our example, it would have the following contents.

*:oracle                cpu,memory      /oracle

Verify that both the cgconfig service and the cgred service are starting from the runlevels. If so, your services will now automatically start in the appropriate cgroup.

Now that you know how to create cgroups and assign resources to them you can more efficiently manage your server resources.

About the author:
Sander van Vugt is an independent trainer and consultant living in the Netherlands. Van Vugt is an expert in Linux high availability, virtualization and performance and has completed several projects that implement all three. Sander is also a regular speaker on many Linux conferences all over the world. He is also the writer of various Linux-related books, such as Beginning the Linux Command Line, Beginning Ubuntu Server Administration and Pro Ubuntu Server Administration.

Dig Deeper on Linux servers