Today's servers come in a variety of flavors, performance levels and prices, but figuring out value to your specific operations poses a difficult question for admins.
A challenge for any data center administrator is figuring the return on investment on a server. Will a 16-core server run my application fast enough to justify the steep price compared to fewer-core models?
This is a tough question. A server's performance isn't just driven by core count and gigahertz: Memory size, cache size and disk I/O have a major influence on performance, as does network speed in many use cases. A server benchmark enables better side-by-side comparisons of real-world performance than datasheets from competitive vendors.
Assuming we know what we need to benchmark -- usually an existing application used by the business -- the hard part is finding a server to run tests on.
The first step is to check if you need to run your own server benchmark. Software vendors, especially in high-performance computing, like to know what their applications can achieve in terms of performance, and may already recommend server configurations complete with benchmark results.
For popular apps, the server vendor may already have a set of numbers from its own benchmarking lab. Compare these results from as evenly matched benchmark tests on competitive systems as possible. On most x64 machines, the processors derive from the same Intel/AMD reference design, so the same configuration benchmarks almost identically. In this case, the benchmark can come from a different server vendor than the one with the best deal.
Without vendor benchmarks, you need to get a server or cluster. The vendor might provide a loaner system for benchmark tests if your total purchase is priced high for a low volume of systems, such as a server loaded with solid-state direct-attached storage. If you're buying just a handful of average-cost servers, consider calling upon a long-standing relationship with the vendor or their channel representative.
Beyond a physical machine to benchmark, the cloud is an obvious place to look. High-core count instances are hard to find on cloud hosts, and the likelihood of finding the exact server model that you want to buy is low. If current-generation machines that are close to your prospective purchase are available as cloud instances, use one of those and extrapolate from the results. For example, Nvidia's GPU instances are available as a service.
About the author:
Jim O'Reilly is a consultant focused on storage and cloud computing. He was vice president of engineering at Germane Systems, where he created ruggedized servers and storage for the U.S. submarine fleet. He has also held senior management positions at SGI/Rackable and Verari; was CEO at startups Scalant and CDS; headed operations at PC Brand and Metalithic; and led major divisions of Memorex-Telex and NCR, where his team developed the first SCSI ASIC, now in the Smithsonian.
Benchmark testing best practices
Big Blue benchmark tests SSDs