Problem solve Get help with specific problems with your technologies, process and projects.

Rootkit levels of infection and mitigation

PaX team developers Ed Tittel and Justin Korelc describe St. Jude and St. Michael intrusion detection systems as coutermeasures against hackers' exploitive use of rootkit.

Hackers don many disguises in order to sneak past IT security guards. The rootkit, one of the most effective disguises, not only masks the intruder, but covers his trail.

The rootkit's origins are deeply rooted in early methods of "backdooring" Unix-based workstations and servers. Current examples encompass a variety of functions and features that further improve upon existing methods (where they don't redefine them outright). Today, the term rootkit is divorced from operating system dependency. While a strong security implementation can help mitigate the effectiveness of rootkit installation, removal of such malware is -- unfortunately -- an inexact science, and usually requires a drive format and full re-installation of the original operating system to ensure a clean and proper restoration.

In its most basic form, a rootkit aims to disguise the presence or activities of a person or process on a target host while providing surreptitious access for later re-entry. Originally, precompiled binaries (designed to conceal certain files or information reported in system utilities) replace original system binaries in attempts to avoid detection. These are commonly known as Trojan programs.

A rootkit might also attempt to patch certain functionality somewhere within an operating system, such as a Linux Loadable Kernel Module (LKM) that cripples security measures for any person or process who possesses proper credentials (this could be anything: a specific user ID, a known password, access to a hidden resource or filesystem path, a predetermined series of events, or even a rarely-utilized system call used in an offbeat manner).

Newer rootkit designs employ more advanced techniques, the scope of which is too broad to cover here. For excellent coverage of this and other related topics, however, we recommend Greg Hoglund and James Butler's Rootkits: Subverting the Windows Kernel (Addison-Wesley, July 2005, ISBN: 0321294319). Though it is Windows oriented, the book still contains a wealth of great general information on rootkits, as does Hoglund's equally good Web site,

Intel-based architectures utilize a logical concept of segmentation called rings to define access control for all processes on a given machine. Rings 0 and 3 are used most often, and correspond to kernel-mode (most privileged) and user-mode (least privileged) operations. An LKM-based rootkit operates within Ring 0, where all the highest privileges apply over the entire system. Common applications (not LKM-based) operate within Ring 3, which depend on interfaces provided by Ring 0 applications or services.

Understanding this basic principle illustrates precisely why attempts to remove a determined rootkit can be impractical. The very tools used to identify such threats are susceptible to direct manipulation by these threats. One such popular tool used for rootkit detection is chkrootkit[1], which can currently identify 60 known Linux-based rootkits.

# tar -zxvf chkrootkit.tar.gz
# cd chkrootkit-0.46 && make sense
# ./chkrootkit
ROOTDIR is `/'
Checking `amd'... not found
Checking `basename'... not infected
Checking `pop3'... not found
Checking `ps'... INFECTED
Checking `pstree'... INFECTED

Searching for t0rn's default files and dirs… nothing found
Searching for t0rn's v8 defaults… nothing found

eth0 is not promisc

Another rootkit detector, implemented at the kernel-level, is part of the St. Jude Intrusion Detection System (IDS), called St. Michael. This module provides preventative and reactive countermeasures against rootkits. For example, St. Michael disables writes to /dev/kmem (effectively preventing attackers from directly overwriting key data structures that affect the entire system), and by system call interposition to monitor activities of all processes on the target machine.

While the St. Jude/St. Michael kernel-level security paradigm offers greater protection from an administrative standpoint, it's no silver bullet in the campaign against hostile events that attack trusted resources. A clever rootkit installed before the very mechanisms that should safeguard against its activities could potentially manipulate the findings of these security tools. Diligent administration and automated self-checking is the only way to keep a healthy network free of the hazardous effects that rootkits can cause.

About the authors: Ed Tittel is a full-time freelance writer and trainer based in Austin, TX, who specializes in markup languages, information security, and IT certification. Justin Korelc is a long-time Linux hacker who works with Ed, and concentrates on hardware and software security topics. Together, the two have recently authored a book on Home Theater PCs and Tom's Hardware 2005 Holiday Buyer's Guide.

For more information see:

  1. chkrootkit
  2. LKMs: St. Jude & St. Michael
  3. St. Jude Model
  4. On Incident Resiliency

Dig Deeper on Linux servers