The measurement of power usage effectiveness (PUE) has provided data centers with a common standard for comparing and improving power usage. But the standard provides lots of room for interpretation, allowing many organizations to slant results to their own advantage. In this tip, we’ll examine the potential flaws in PUE and explain how the more recent PUE 2 standard can close some of these loopholes.
The flaws in PUE
PUE was meant to be a means of evaluating and tracking how effectively our data centers utilize the massive amount of power they consume and help us measure the effectiveness of our facility improvements. But we quickly turned PUE into an “extreme sport” contest, where enterprises published and advertised PUE numbers that almost no one believed. No one provided details as to how the numbers were obtained or where measurements were taken, so we didn’t know how they were derived or what they really meant. And PUE is a very easy number to manipulate.
On October 29, 2009 The Green Grid released Version 2.1 of their PUE white paper, which people now refer to simply as “PUE v2.” It was meant to provide a more accurate means of reporting PUE, but confusion still remained. On May 17, 2011, the Data Center Efficiency Task Force (DCETF) published a paper that clarified the recommended means for measuring and reporting overall PUE.
What has PUE 2 clarified, and what PUE reporting errors is it supposed to correct?
As most everyone knows by now, the PUE definition is deceptively simple. It is the ratio of total power going into the data center divided by the power actually used by the IT computing equipment.
Total data center power in
IT load power
A PUE of 1.0 would be perfect, and would mean that all incoming power is going to the IT equipment. It would mean no cooling; no lights; not even wiring, in which some power loss always occurs. That’s technically impossible. Most legacy data centers are probably have PUE levels between 2.5 and 3.5, but claims have been made of new data centers and even containerized structures with PUE’s less than 1.1. That’s pretty amazing!
So what makes a PUE number really low? Simple: reducing the numerator or increasing the denominator makes the PUE quotient smaller. Like a golf score, a lower number is supposed to say the data center is better. But golfers can cheat, and PUE numbers can be manipulated. How? For starters, you might take instantaneous power draw measurements at your best operating time, when it’s cool outside, all the lights are off, but your computers are cranking out research or most of your customers are online. You might even shut down redundant cooling for a few minutes while you take readings. The result could be a really good number, but would it really mean anything?
There are more ways still to distort the PUE figure. Start with what is included in “Total data center power in.” How about your network operations center (NOC), or all the IT offices? Leave those out and the total power becomes smaller and PUE becomes better. Where do you measure the “IT load power?” If it’s from the read-out on the uninterruptable power supply (UPS) output, then you’ve added power distribution unit (PDU) transformer losses, wiring losses, and things like cabinet fans into your IT load calculation, even though they don’t contribute a thing to the actual processing work. So the denominator gets bigger and your PUE looks even better. And what if you have a shared facility with a common power feed for the building and all its offices, conference rooms, food service and lobbies? Or a shared chiller plant or cooling tower? If they’re not all separately metered, some “creative estimating” of the portion of incoming power supposedly used by the data center can dramatically affect your PUE.
The improvements in PUE 2
In short, PUE was not originally intended to compare dissimilar data centers. It was meant as a means of tracking how an individual data center is performing over a period of time. In this case, there would be no benefit at all to manipulating the numbers, since the purpose was strictly for internal improvement. The Green Grid maintains that PUE is not meant to compare different facilities – but since people are going to make comparisons anyway, it should at least be done on a uniform basis, with all the right measurements included. Enter “PUE v2.”
PUE Version 2 makes four major and very important changes in the PUE metric:
- PUE v2 establishes different types of PUE measurement. The Green Grid calls them Levels 1, 2 and 3; and labels them Basic, Intermediate and Advanced. The DCETF adds a fourth even more basic level, and identifies the categories as PUE0 to PUE3. Each higher level or category is more rigid and exacting than the one before it, and it is now required that any stated PUE value identify the measurement level or category that was used to derive it. For the rest of this article, we will discuss the DCETF categories of PUE0, PUE1, PUE2, and PUE3.
- With the exception of PUE0, all measurements must now be converted to energy, which means an included time component like kilowatts per hour instead of just kilowatts. And it must include all incoming energy sources, corrected to a common equivalent (e.g., the gas burned for heating must be equated to kilowatts per hour.). Further, all measurements must be made at specified points for each category, and taken over at least a one-year time frame.
- PUE v2 now stipulates what amenities (such as offices and NOCs) must be included in the PUE calculation for different types of facilities (e.g., data centers, containerized assemblies, and so on) – if it’s IT-related, it’s in.
- PUE v2 also makes clear that renewable energy sources, such as wind and solar – while certainly valuable ways of reducing carbon footprint – have no bearing or influence on PUE. The PUE metric is meant strictly to quantify the efficiency with which the compute facility utilizes the energy it consumes, regardless of where or how that energy originated.
Let’s briefly summarize the differences among the four new PUE categories to better understand what is necessary to meet each level, and why PUE measurements in different categories shouldn’t be compared.
PUE Category 0 (“PUE0”): This is essentially the original PUE, in that it still measures “Total data center power in” and “IT load power” at discrete points in time. The major improvement is that it now specifies that the readings be taken “during peak IT equipment utilization.” Therefore, PUE0 does not show the effects of varying loads, so may still be significantly skewed depending on when measurements are made. Further, “IT load power” is taken at the output of the UPS, so will add any non-IT overhead such as PDU transformer losses and cabinet fans to the real IT load. And PUE0 is still based on power rather than energy, so it can only be used for data centers that are 100% electrically powered. There is no means of correcting for different fuels. In short, it can be useful for tracking the effects of changes in an individual data center if measurements are taken consistently and compared realistically. But PUE0 should never be used to compare different data centers, even if they are supposedly of the same designs.
PUE Category 1 (“PUE1”): This is the first of the new PUEs based on energy or “power consumption” calculations. It requires that readings be made as total kilowatts per hour over a 12-month period, and that all fuel types serving the data center be converted to a common value – usually to the equivalent kilowatts per hour. This is a major improvement over PUE0, but still measures the “IT load energy” at the UPS output, so it retains the same potential for errors in IT load that were noted for PUE0. Depending on actual conditions, these errors could skew the PUE to a somewhat better number than reality. But if there are no PDU transformers, and no cabinet fans, and everything else is measured in full accordance with the PUE1 methodology, it will probably be a pretty accurate number.
PUE Category 2 (“PUE2”): The only real difference between PUE2 and PUE1 is that the “IT load energy” is measured and summed from the outputs of the PDUs. As noted above, if PDUs with transformers are not utilized in the power delivery chain, there should be very little difference between PUE1 and PUE2 numbers.
PUE Category 3 (“PUE3”): This is the ultimate and most accurate PUE measurement method, and the level to which every data center should ultimately aspire to use. It requires having the infrastructure metering necessary to accurately measure cumulative IT load energy over a 12-month period at the power input connection of each IT device. Very few data centers as yet have the ability to do this, but it is certainly the most accurate approach. In the opinion of this author, no one should claim a PUE better than 1.3 unless it was developed in full compliance with the requirements of PUE Category 3.
Don’t mix PUE categories
So now we should now be seeing PUE numbers that have real meaning – or at least that we know were supposed to have been derived in a way that enables us to assess them with understanding. But the different measurement points specified for each of the PUE 2 categories also show us how meaningless it would be to compare a number derived under one category, such as PUE1, with one derived in accordance with requirements of another category, such as PUE3. They would be “apples to oranges” relationships.
PUE Version 2 removes the ambiguities and suspicions that have been associated with many PUE claims in recent years. It also defines enough different measurement methods to make it possible for most anyone to calculate their PUE, regardless of the level of measurement sophistication they have available. But it also ensures that when PUE numbers are reported, there is an associated understanding of how they were supposed to have been developed, and how close they may be to the kind of truly accurate PUE number that would come from a PUE3 calculation.
But PUE v2 also makes clear that establishing a meaningful PUE number under any of the four categories requires more than just reading the UPS and the building electric meter. A significant amount of accurate sub-metering or shared load allocation is necessary to obtain the required information. For new or renovated data centers, a full range of metering should be incorporated into every design so that data centers can be truly responsible about their energy usage and efficiency.
Robert McFarlane is a principal in charge of data center design for the international consulting firm Shen Milsom &Wilke LLC. McFarlane has spent more than 35 years in communications consulting, has experience in every segment of the data center industry and was a pioneer in developing the field of building cable design. McFarlane also teaches the data center facilities course in the Marist College Institute for Data Center Professional program, is a data center power and cooling expert, is widely published, speaks at many industry seminars and is a corresponding member of ASHRAE TC9.9 which publishes a wide range of industry guidelines.