This tip is part four of a series on using data center infrastructure management tools. Read part three: Demystifying energy-monitoring hardware for effective DCIM.
Besides power data and energy use, environmental information is also collected by the DCIM systems to help optimize cooling -- either from various temperature and humidity probes located throughout the data center or at the rack. In some cases, several of the newer “intelligent” rack power distribution units (iPDU) have integrated these sensors so that every rack can be monitored (in high-density applications, up to three temperature sensors per rack). Virtually every computer room air conditioning (CRAC) or computer room air handler (CRAH) unit measures the temperature of the return air, however, almost none of them measure the output airflow. Therefore, it is preferable to also install a temperature probe at the output. This additional information can also help analyze and balance which cooling units are receiving and meeting the heat loads.
Frequency of data gathering
One question often asked is: What is the recommend polling interval for energy use monitoring? The polling interval is usually controlled by the DCIM software, however, there is no simple answer. Continuous monitoring offers the best accuracy, and one to 15 minutes is typically considered nearly continuous, while others find hourly monitoring quite acceptable. Daily monitoring is satisfactory for annualized energy use calculations. However, it is important to recognize that what is being monitored and how much processing is being done at the sensor level will affect the accuracy and integrity of the measurement.
Energy vs. power measurement
The terms power and energy are often used interchangeably by many people, while in fact, they are not the same. Energy, expressed as kilowatts per hour, is defined as the power (kilowatt) used over time (i.e., 10 kW used for one hour equals 10 kWh). This fact makes a big difference if the instrumentation is only instantaneously measuring power (kilowatt) or continuously recording kilowatts per hour. For example, by definition, a sensor that measures energy (kWh) is presumably an intelligent unit, recording power continuously and providing an accurate picture of the energy use, regardless of the polling interval of the DCIM platform.
However, a device that can only instantaneously measure kilovolt-amperes or kilowatts will only report a snapshot reading of what it measured at a given time. This may be good enough for loads that are relatively constant and can be polled hourly and then averaged over time. However, for cooling equipment such as chillers and CRACs, which contain compressors that cycle on and off, just reading instantaneous kilovolt-amperes or kilowatts may provide meaningless readings depending on when the sensor was polled, and whether the compressor was on or off at the time.
Measuring other forms of energy
We normally think of energy monitoring in terms of electricity and kilowatts per hour, but energy also comes in the form of heat (or heat removal), which is measured in BTUs. In some cases, a data center may not have its own dedicated chiller plant and it may receive chilled water or dump excess heat into a building-based “condenser loop” or glycol loop. In those cases, it is not normally possible to directly measure all the electrical energy used to cool the data center.
The alternate method is to measure the amount of BTUs removed by the chilled water or glycol loop. This can be accomplished by measuring the number of gallons of fluid passing into and out of the data center CRAC or CRAHs and the difference in the fluid temperature (delta T) to calculate the heat energy. The temperature can usually be measured non-intrusively by sensors placed in direct contact with the outside of the piping, although it is more accurate to have probes that penetrate the piping into the fluid. Measuring fluid flow can require installing an in-line flow sensor into an existing fluid system. This is an intrusive procedure and requires shutdown or bypass of the piping. As you can imagine, this procedure is not looked upon favorably in a data center. Some vendors have introduced non-intrusive ultrasonic flow sensors that can be installed on the outside of the pipe, but these require careful calibration to provide accurate fluid-flow readings.
Some of the more advanced data center efficiency scenarios involve the reuse of the heat energy normally wasted by the cooling system. In those cases, the same type of sensors and methodology can be used to measure the recovered energy. While energy recovery is specifically excluded under the updated power usage effectiveness (PUE) guidelines, The Green Grid has recently developed the Energy Reuse Effectiveness (ERE) metric for that specific purpose.
Input energy to the data center from fuels such as natural gas (if used for local co-generation) are calculated in BTUs and need to be accounted for in total efficiency calculations. This has also contributed to the recently issued update of the PUE as a global metric
The Bottom Line
Clearly, the installation of DCIM energy-monitoring hardware requires the cohesive efforts of the IT and facilities departments. While it would be best if energy monitoring was part of the original data center design and pre-installed when the site was built, energy efficiency and measurement has only recently become a high priority in the data center. A higher level of granularity directly affects the cost and complexity of the DCIM installation. This is especially true in the case of existing sites, due to the significant retrofit labor cost of installation, as well as the hardware cost.
Nonetheless, the majority of existing data centers lack the instrumentation to accurately monitor energy use of the disparate systems individually and organizations are beginning to address this. The challenge is to have facilities and IT cooperate and collaborate to properly plan and manage the installation of the energy-measurement hardware, as well as to integrate existing building management systems and its collected data into the DCIM platform.
ABOUT THE AUTHOR: Julius Neudorfer has been CTO and a founding principal of NAAT since its inception in 1987. He has designed and managed communications and data systems projects for both commercial clients and government customers.