This tip is the second in a series on measuring server performance. Read part one on best
practices for server benchmark testing and part three on server
benchmarking tools and stress testing.
The concept of benchmark
tests is easy, but using a benchmark and obtaining meaningful data that can actually improve
system performance is another matter entirely. Benchmark tools, such as Microsoft's Performance Monitor
(PerfMon), are quite flexible, but the variety of counters and settings available within a tool can
complicate testing and make results difficult to interpret. In this tip, we'll examine some of the
most common counters used with PerfMon and see how they might influence a benchmark test.
Memory allocation and general RAM settings
Allocating too much RAM to an application can actually hurt performance of other processes on
the machine. In fact, improper memory utilization can lead to negative system performance.
When conducting internal server benchmark tests with PerfMon,
use the following counters to verify that memory allocation will not disrupt your server
performance:
- Memory:Available Bytes--This counter measures the total physical
memory available to the operating system (OS) and compares it with the memory required to run all
of the processes and applications on your server.
- Memory:Committed Bytes--This comparison should be tracked over time to
allow for periods of peak activity. You’ll want to monitor when there is a peak and a valley in
Committed Bytes to understand how your machine is performing. You should always have at least 4 MB
or 5% more of available memory than committed memory.
- Memory:Page Faults/sec--This counter measures page faults that occur
when an application attempts to read from a virtual memory location that is marked "not present."
For the most part, zero is the optimum measurement. Any measurement higher than zero delays
response time. Remember, the Memory:Page Faults/sec counter measures both hard page faults and soft
page faults. Hard page faults occur when a file has to be retrieved from a hard disk rather than
virtual memory. On the other hand, soft page faults occur when a resolved page fault, which is
found elsewhere in physical memory, interrupts the processor but has much less effect on
performance.
Thread and process monitoring
Pay attention to several of your processor counters, particularly when you're trying to
maximize the number of threads per processor. Look at the number of “context switches” that
occur.
A “context switch” occurs when the kernel, or core of the OS, switches the processor from one
thread to another. Context switches should be avoided, as each context switch causes the processor
L1 and L2 caches to be flushed and refilled. Cache flushing and refilling takes precious time and
reduces the system's performance.
- Process:Thread Count:Inetinfo--Counts the number of threads created by
the Inetinfo process and displays the most recent value.
- Thread:% Processor Time:Inetinfo =>Thread #--Measures how much
processor time each thread of the Inetinfo process is using.
- Thread:Context Switches:sec:Inetinfo =>Thread#--Measures the
maximum number of threads per processor or thread pool. Monitoring this counter is important to
prevent creating so many context switches that the memory being lost to context switches supersedes
the benefit of the added threads, at which point performance will decrease rather than
improve.
Measure and analysis
Unfortunately, there is an extraordinarily broad range of processes and server aspects that can
be measured–far too many to list in this article. However, for the most part, system performance
and metric testing can be categorized into the following groups:
- Memory management
- Network capacity
- Processor capacity
- Disk optimization
By looking at the above groups, a testing engineer should get some solid benchmark results that
they can utilize to better enhance the overall server environment.
Meeting the challenges of benchmark tests
As with any testing done in a server environment, benchmarking and metric evaluations come with
some notes of caution.
- Be wary of vendor-provided benchmark results. Vendors tend to tune
their products specifically for industry-standard benchmark results. This means that the benchmark
documentation on paper may not suite your unique environment. For example, let’s assume that an IT
manager is about to purchase an application that is going to handle a user database stored on a
server. Metrics for that application show it running well on a Server 2008 box with quick response
times. Although this may sound good, it may not necessarily fit with what your environment is
running. For example, what if the application that was tested by the vendor was running on a
single, beefed up server while yours is on a VM which will be sharing resources? Remember, the goal
of the vendor is to sell you the software, so some are known to “cheat” their numbers a bit. Doing
so improves the numbers on paper, but might make things a lot worse in a live environment. Although
not as common amongst large hardware and software vendors, smaller resellers have been known to
slightly alter their numerical data. For example, a hardware device capable of delivering VPN
connectivity over the WAN may have had better delivery times because they utilized optimized
hardware for the benchmark test. However, when deployed onsite, the devices showed a solid 20%-30%
reduction in speeds. So, due diligence is always recommended with devices or applications that are
going to be heavily relied upon.
- Never focus on just one benchmark. When conducting server benchmark
tests, try to include as many components as possible. Don’t focus on just one element, like CPU
speed. Watching the behavior of various key elements in the server allows an engineer to gain a
better overall understanding of how the server performs under various conditions, which can help in
tracking down and correcting future performance issues.
- Be careful of benchmark service providers. If you’re planning on
outsourcing the benchmark and metric testing, make sure to do your due diligence. Many times, even
reputable consulting firms either disregard or do not follow basic scientific methods. This
includes, but is not limited to, small server and application sample size, lack of variable
control, the limited repeatability of results and working with software and hardware numerical
biases. Look for numerical extremes, such as a SQL server running way better than it should be,
based on the hardware used. Vague hardware specifications are also a red flag. If a vendor lists
the hardware but doesn’t give any detail on it--Dual Core Processor, 4 GB RAM, 512 MB video card--
you should be careful. When figuring out minute details of benchmarking, every variable matters. In
this case, what type of processor was it? What model RAM was used and what model video card was
installed? All of these details make a difference.
-
The key point to note is that every environment is unique and has its own set of requirements.
Metric testing with tools similar to PerfMon is an ongoing process with numerous variables that
will unquestionably affect the results of your data. By planning out the test and following a solid
scientific method, the testing admin can more accurately gauge how hardware and software are
performing. When done properly, a good benchmark analysis can provide information that improves
server architecture performance.
ABOUT THE AUTHOR: Bill Kleyman, MBA, MISM, is an avid technologist with
experience in network infrastructure management. His engineering work includes large virtualization
deployments as well as business network design and implementation. Currently, he is the Director of
Technology at World Wide Fittings Inc., a global manufacturing firm with locations in China, Europe
and the United States.