TPC eyes energy consumption and virtualization benchmarks |
 |
By Bridget Botelho, News Writer
06 Nov 2008 | SearchDataCenter.com |
|
|
Last month, the San Francisco-based nonprofit Transaction Processing Performance Council (TPC) marked its 20-year anniversary by offering vendor-neutral processing performance benchmarks, disclosing plans for future benchmarks and offering a workshop for end users along the way.
Currently, TPC's four active benchmarks are TPC-C and TPC-E for online transaction processing, TPC-H for decision support for ad hoc queries and TPC-App for business-to-business transactional Web services.
And now, in an effort to keep pace with data center initiatives to improve energy consumption and to green IT, TPC plans to offer benchmarks that include energy consumption metrics.
"TPC is currently working on a specification for how to measure and report energy consumption within existing TPC performance benchmarks," said Mike Molloy TPC's chairman. "TPC-Energy will measure the total energy to complete a certain amount of computational work. It will also allow users to measure the power consumed when systems are idle."
TPC will also offer new ETL (extraction/transformation/loading and service-oriented architecture (SOA) measurements. Further, the organization plans to offer a series of workshops on using performance benchmarks to make server hardware and software buying decisions. The workshops will be held in various locations throughout the U.S., and the first workshop begins Nov. 19 in Dana Point, Calif.
Tackling energy consumption
TPC now examines energy issues that include handling multiple power measurements points and handling reporting when only some of a set of identical subsystems are instrumented and when certain kinds of instrumentation are used. TPC has also explored ways to correctly report energy measurements during separate phases of a benchmark as well as subsystem reporting and the impact on the primary metric (either watts per perf or perf per watts), Molloy explained.
Similarly, last December, the Warrenton, Va.-based Standard Performance Evaluation Corp. published the SPECpower-ssj2008 benchmark to compare a server's power consumption with its performance. SPEC has also been at work on a virtualization performance benchmark since 2006
TPC's SOA benchmark is only in the proposal stage, but the tentative plan is to focus on common industry-accepted portions of SOA infrastructure, mainly Web services, the enterprise service bus, and business process choreography. As advanced SOA practices become more standard in the industry, TPC will expand the benchmark to incorporate additional SOA infrastructural services, Molloy said.
The ETL benchmark is currently in the proposal stage as well. The tentative plan is to model multiple data sources, including an OLTP system, along with a dimensional data warehouse model for a destination database. The benchmark will specify the transformations required to perform an update of the target database from the sources, and measure the performance of the transformation and loading process, Molloy said.
Benchmarking in a virtual world
Future TPC benchmarks will also include guidelines for measuring workloads in virtual environments. TPC hopes to devise a measure of virtual server performance in the same way physical servers are measured.
"There is no reason these benchmarks can't be run in a virtualized environment, and most of our benchmarks will include guidelines on how to measure workloads in virtual environments in the next updates," Molloy said. "With virtual servers, you can have more than one on a physical server, so reporting the performance for all the virtual servers on a physical server is where the rules must be defined."
To date, however only virtualization provider VMware Inc. offers a virtualization benchmarking system, which is called VMmark. TPC's benchmarking system will be more flexible than VMmark, according to Molloy. "VMmark requires a fixed set of applications and guest OS to be run (called a Tile). That configuration is not allowed to be changed. We would allow different combinations and a number of applications/Guest OS," Molloy said.
Let us know what you think about the story; email Bridget Botelho, News Writer. And check out our data center blogs: Server Farming, Mainframe Propellerhead, and Data Center Facilities Pro.
|
|
|
|
