Since 1988, the Transaction Processing Performance Council (TPC) has created benchmarks to measure the performance of full systems that execute transaction processing. The organization has developed nine benchmarks, each addressing distinct requirements
Performance and price/performance metrics are key criteria in data center purchasing decisions,
Demand for a new energy metric
The unprecedented growth in reliance on computers (and Internet) to run the world's industries and governments has led to an explosion in server installations, both in size and number, as well as the amount of energy required to operate and cool them. Energy consumption has increased exponentially over recent years -- a trend that will continue to accelerate into the future. This is evidenced by the Environmental Protection Agency's report, "EPA Report to Congress on Server and Data Center Energy Efficiency," in which data center energy consumption within the U.S. is projected to surpass 100 billion kWh by 2011, with an annual electricity cost of $7.4 billion.
The requirement to reduce energy costs and usage and satisfy the mounting demand for additional computing resources has become the greatest challenge for many IT organizations. Data center growth is constrained by hard limits on energy consumption due to facility issues, limitations of the power grid and/or policy decisions. Public awareness of data center energy consumption and its impact on the environment has influenced many companies to place a higher priority on choosing "greener" technologies to "do their part" to protect the environment.
Solving the data center energy consumption dilemma
What dilemma? Well, for one, doubling or tripling the computing performance without increasing energy consumption. This is a common theme heard in corporations around the world -- "do more with less." More computing power with less energy consumption, less heat generated and less cost! The three most important criteria are addressed in this dilemma: computing performance, energy consumption and price. Each of these forces pull in different directions. As forces interact in physics, so too they interact in this dilemma. These forces are causing IT decision makers to deploy technologies that meet these new demands.
How will TPC energy benchmarks help?
Having a standalone measure of energy consumption without also incorporating performance and price is like providing only the miles-per-gallon rating of a vehicle. Without knowing the speed, size or price of the vehicle, it is unlikely that a good business decision can be made about its purchase. Additional data must be provided before a well-informed purchase can be made. Does the vehicle (computing system) provide the performance required? Will the vehicle (computing system) allow the users to meet his requirements? Is the price versus these other requirements acceptable? The TPC Energy metrics will provide the additional dimension to the computing systems' performance and price. As in the TPC's price/performance metrics, which rank computing systems according to their cost per performance rates (e.g., TPC-C $/tpmC), the TPC Energy metrics will rank systems according to their energy consumption per performance rates. This will be in the form of watts/performance rate (e.g., watts/KtpmC). The ranking of the Top Ten energy/performance systems will be available on the TPC website.
The three most important criteria in IT purchases are performance, price and energy consumption. But today's complex IT environment demands that price and energy consumption be put in perspective of performance. Reducing costs or energy consumption at the expense of performance is often unacceptable. The TPC metrics of price/performance and energy/performance addresses this concern. Customers increasingly require these metrics to be provided for IT purchasing decisions.
Buyers now demand an objective method of comparing all three factors to select equipment that best meets their changing requirements, and the TPC's Energy Specification is being carefully designed to address this need. Like the TPC Pricing Specification, the TPC Energy Specification is a supplement to existing TPC benchmark standards, rather than a standalone measurement framework. This means that it is intended to be compatible with TPC benchmark standards currently in use, including TPC-App, TPC-C, TPC-E and TPC-H. The result will be metrics that enable comparison of systems on all three axes -- price, performance and energy consumption.
Benchmarks in 2010 and beyond
While the TPC-Energy metric is intended to help buyers identify equipment based on their price, performance and energy consumption requirements, TPC is also looking to the future of benchmarks in 2010 and beyond.
To this end, TPC began reaching out to industry experts and analysts in April 2009 through a call for papers to be presented at the first Technology Conference on Performance Evaluation and Benchmarking (TPC TC 2009). We have received an overwhelming response from industry experts and academia interested in influencing future benchmark developments.
The TPC encourages interested readers to attend the workshop, which will be held on August 24 in conjunction with the VLDB 2009 premier annual international forum for database researchers, vendors, practitioners, application developers and users. Additional information about the conference and registration are available at http://www.vldb2009.org/ and http://www.vldb2009.org/?q=node/7.
TPC TC 2009 contacts:
Raghunath Nambiar, General Chair, email@example.com
Meikel Poess, Program Committee Chair, firstname.lastname@example.org
Nicholas Wakou, Publicity Committee Chair, email@example.com
ABOUT THE AUTHOR: Mike Nikolaiev is the chairman of TPC's Energy Subcommittee and Director of ISS Performance Engineering at Hewlett-Packard Co.
What did you think of this feature? Write to SearchDataCenter.com's Matt Stansberry about your data center concerns at firstname.lastname@example.org.
This was first published in June 2009