"Power has become [so important] in the last few years," said Greg Darnell, the vice chairman of the SPEC power committee who also works at Dell Inc.'s server performance analysis lab. "It was an obvious choice for SPEC to pursue this as one of the leaders in industry-standard benchmarks. We prefer to get together as a consortium of industry vendors and come up with something that everyone can agree on."
For about a year, the nonprofit organization has deliberated over the metric and has accumulated about a dozen results on the benchmark's page, including Intel Corp.'s Xeon -based systems from Dell Inc., Hewlett-Packard Co. and Fujitsu Siemens Computers.
SPECpower-ssj2008 is expressed as a performance-to-power ratio, so the higher the number, the better. To gauge performance, the benchmark measures how a server executes a standard Java workload at CPU load levels in 10% increments from 0% to 100%. The results are published in text and graphical form so users can see a chart of how each server performs. Here is a graphical example from an Intel Xeon-based HP ProLiant DL160:
Deciding how to measure the performance side of the ratio was easy, Darnell said: SPEC has simply built off the existing Java workload benchmark. But how to measure power consumption was less obvious. Ultimately, SPEC approved three "power analyzer" devices -- the Yokogawa WT210, ZES LMG450 1-channel and the CLM3000 1-channel -- that measure a server's power draw. It also specified that the ambient temperature in the room be at least 20 degrees Celsius (or 68 degrees Fahrenheit), which is the low end of the range recommended by the American Society of Heating, Refrigerating and Air-Conditioning Engineers Inc. (ASHRAE). That temperature must be confirmed by one of two SPEC-approved temperature sensors: the Digi Watchport/H or the Temperature@lert.Another energy efficiency effort
The past couple of years have been marked by a lot of collaboration and discussion about data center energy efficiency, of which server efficiency is one part. Some of the major groups involved are SPEC, Green Grid, the Uptime Institute Inc., and the federal Environmental Protection Agency (EPA).
Green Grid and the Uptime Institute -- both of which have developed metrics that compare the amount of power a data center consumes with how much power gets to the IT equipment – look at data centers holistically. Of the two, the Green Grid's Power Usage Effectiveness (PUE), appears to be gaining traction, but it's still early to say so definitively. The EPA said that it and the federal Department of Energy will also release data center efficiency metrics sometime in 2008
Meanwhile, the EPA has considered the development of an Energy Star label for servers much like the one established for PCs, ceiling fans and washing machines. The SPEC performance-power benchmark might be one of the foundations for that Energy Star label.
"The ability to measure power consumption in a consistent way across multiple server platforms is an important element of the EPA Energy Star program," Andrew Fanara, the director of the EPA's Energy Star Product Specifications Development team, said in a written statement. "SPEC has taken a critical first step to give server vendors and their customers a standardized benchmark tool that elevates power efficiency in the performance evaluation process."
Reached by phone yesterday, Fanara said the EPA hopes to put out a first draft of the Energy Star label for servers by the end of January. The SPEC benchmark could play a role there.
"We encourage them to use this as part of their Energy Star label," Darnell said. "But we would encourage them to not use it as the only measurement, because it's only measuring one workload."
Darnell added that SPEC will publish similar performance-to-power benchmarks in the future using workloads other than Java.