This tip goes deeper into the Net-SNMP systems management agent, which was discussed briefly in an earlier tip focused on OpenNMS.

Most modern distributions of GNU/Linux include a powerful systems management agent called Net-SNMP. The power of this agent sometimes feels inaccessible to the average systems administrator who may not be familiar with Simple Network Management Protocol (SNMP). SNMP has come to symbolize the entirety of the Internet Engineering Task Force's (IETF) systems management framework. This tip is far too short to present a comprehensive overview of SNMP, but it explains why it is important to learn more about the Net-SNMP agent.

One important difference between SNMP and other common TCP/IP protocols that system administrators often work with is its protocol formatting. Instead of human-readable, ASCII-coded commands, all communication is in a binary format that is optimized for efficiency of size. Another differentiation that can make SNMP feel somewhat obtuse is that on IP networks,

it uses the connectionless User Datagram Protocol as its default underlying transport protocol, rather than the more familiar, connection-oriented TCP. The combination of these two means that unlike Simple Mail Transfer Protocol (SMTP) or HTTP, you can't just use a tool like Telnet to communicate via SNMP to a remote system for troubleshooting or educational purposes.

The Net-SNMP project includes a set of command-line tools that are usually bundled separately from the agent. On Red Hat Enterprise Linux and similar distributions, they are in a package called Net-SNMP tools. Of course, you'll also need the agent component, which is packaged simply as Net-SNMP. For a set of instructions on minimally configuring the Net-SNMP agent, see my earlier tip.

A brief history of Net-SNMP
The Net-SNMP project began in the late 1980s at Carnegie Mellon University and Hewlett-Packard Corp. The project was loosely referred to as “CMU SNMP," and it ran on many popular proprietary UNIX systems, including HP-UX and SunOS 4.x. Later, the University of California, Davis, took over stewardship of the project, renaming it to UCD-SNMP, a name that can still be spotted in some of the management information bases (MIBs) that the agent supports. The Net-SNMP moniker resulted from the team of maintainers becoming gradually less connected to the university after the project's lead maintainer, Wes Hardaker, left to work in a private industry. Today, a worldwide team of coders led by Hardaker collaboratively maintains Net-SNMP as a modern open source,software project.

The latest popular distributions bundle version 5.5 or 5.6 of Net-SNMP. These versions are by far the most feature-filled and interoperable releases in the project's history, thanks to many bug fixes. Earlier versions sometimes exhibit bizarre bugs when communicating with SNMP applications that do not use Net-SNMP's protocol libraries.

What can Net-SNMP do for me?
The Net-SNMP agent provides a huge amount of useful information about the system on which it runs, starting with statistics about the system and its network interfaces. The network interface statistics come primarily from a set of IETF-standard MIBs, including MIB-II and other standard MIBs that extend it. The system-level information about CPU, memory utilization and load average reside in both the IETF Host Resources, MIB and a sysstat MIB group added during the UCD days. Historically, these MIBs have almost always been enabled in the configuration used by the various distributors.

In recent years, it has become increasingly common for the agent to be distributed with support for the Net-SNMP specific MIB tables that hold statistics on file system space utilization (dskTable) and disk input/output (diskIOTable). The dskTable does not contain any entries unless you have added one or more disk directives, or the blanket includeAllDisks directive, to the /etc/snmp/snmpd.conf configuration file. The manual page for this file explains the syntax of these directives.

Once set up, the Net-SNMP agent provides information for each configured file system, including the total, available and used space displayed in kilobytes and percentages. Equivalent statistics for tracking the utilization of file system allocation nodes, sometimes called “inodes” are also available. These latter numbers can be a lifesaver if collected and tracked in an SNMP-capable systems management framework, such as OpenNMS, that supports performance data thresholding. Every seasoned system administrator has at least one war story about running out of inodes on an important file system.

Need more data? Request an extension
If you like what you're learning, but know that the Net-SNMP agent's various built-in MIBs don't provide the information you need on your critical applications, don't assume that you'll have to choose a different technology. The Net-SNMP agent has long supported several mechanisms for extensibility. At the more difficult end of the spectrum, you can write your own extensions or subagents in C or Perl that link directly into Net-SNMP, or communicate with it via the standard AgentX protocol.

If you don't have the time or skills to climb that learning curve, the pass and pass_persist directives use a lightweight protocol to request objects' values from a script or program. By far, the easiest extension mechanism is the aptly-named extend directive, which improves upon the deprecated sh and exec directives.

If you can write a script in the language of your choice that retrieves the data you want in bite-sized pieces, then you can use extend to “glue” that script's output into the Net-SNMP agent's MIB. Net-SNMP caches the output of these scripts for a short time, five seconds by default, to avoid putting too heavy a load on the underlying operating system when many requests for results arrive in quick succession.

Secure in your systems management
Over the years, the SNMP framework has become associated in many people's minds with security problems. This association comes chiefly from the fact that the original version of the SNMP protocol had no support for encrypting the data between managers and agents. The only built-in security was the notion of a plaintext key called a community string that acts like a password. The second version was meant to support strong security, but the working group that designed SNMPv2 had to scrap those improvements when the final reviews of their work uncovered fatal flaws in the security infrastructure. For this reason, the correct term for referring to the second version of SNMP is “SNMPv2c” or “SNMPv2 with community-based security.”

The third version of SNMP, known simply as SNMPv3, does include support for both strong authentication, which resists playback attacks but provides no transport security of its own, and cryptographic privacy to protect SNMP data as it travels on the network. Recently, the Internet Engineering Task Force (IETF) has proposed additional security mechanisms to simplify and improve SNMPv3's security aspects.

In the next tip, we'll explore the practical application of the Net-SNMP agent, particularly its extensibility mechanisms. As you may have guessed, the management framework at the other end of the network will be OpenNMS.

About the author: Jeff Gehlbach has worked since the mid-‘90s in the UNIX and Linux space and since 2000 in the network and systems management industry. Gehlbach’s experience includes implementation, development, consulting and support roles with a wide variety of management frameworks, both proprietary and open source. You can follow Gehlbach's brief musings on more topics on Identica or Twitter.

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.