Skyrocketing energy costs in the data center aren't just a corporate issue. Server consolidation, utilization efficiency...
and virtual machine sprawl also challenge the biggest enterprise in the United States: the federal government.
Over the past several years, the operators of the data center that supports operations at the U.S. House of Representatives has reduced energy consumption by over 50%, saving taxpayers big bucks and allowing the rollout of server virtualization to member offices.
That effort took hard work, but the return in energy efficiency and carbon footprint reduction also helped improve security, disaster recovery and brought the flexibility necessary to enable desktop virtualization in the future.
"Long before there was a green initiative, before it became fashionable, we had a power issue," said Jack Nichols, director of the Enterprise Operations department for the House Information Resources (HIR).
Eight years ago, he explained, the House data center in the Ford office building was "chock full of equipment" and was right near the capacity of finite power that could be delivered to the old structure. After years of right-sizing, consolidation and virtualization, the House data center is now saving nearly $1,000 a day in energy costs, Nichols said. Best of all, "We were able to do this virtualization and consolidation effort without an additional dollar in our budget, using planned lifecycle replacement money."
Nichols, an Air Force veteran, knows a thing or two about working under pressure. He used to be attached to the White House communications agency, where he experienced the unique pleasure of having the President of the United States looking over his shoulder while he set up a mobile network. After a tour through contracting and some time consulting for Lucent, he now manages the IT infrastructure for the House enterprise operations group.
The enterprise ops group had a tactical reason for approaching server consolidation and creating a green data center, Nichols explained in an on-site interview. "We had about 500,000 watts available to us for powering the servers, which was very near capacity," Nichols said. "Another 750,000 watts or so of power was available to us for the servers to be cooled. We needed to look at ways technology could help us reduce consumption."
The group looked at everything available to figure out what was possible. In the end, the group adopted tried-and-true best practices for reducing energy consumption in a green data center. Nichols said that they've reduced energy consumption to between 125-150,000 W, which equates to about $1,000 per day in real savings, assuming a cost of 11-12 cents per kW hour. That adds up to about $365,000 saved annually for taxpayers.
How did they reduce data center power costs?
First of all, test environments weren't being decommissioned. "We inventoried all of the servers to see what could get turned off," Nichols said. After that, they looked at whether servers were matched to engineers' needs, right-sizing servers for workloads. Nichols said that they achieved 45% power reduction in the servers chosen through right-sizing alone.
Consolidation of applications, especially in the data center's Unix environment, was the next target. His group was able to host multiple applications on a single platform, leading to a 55% energy reduction for the relevant machines.
Finally, virtualization was a significant factor. "The biggest savings here was in our test environments," said Nichols. "We realized 75% to 80% power savings there. We had about 200 servers, which we virtualized into a single rack."
Nichols was also able to significantly improve utilization rates. "In our Windows environment, for instance, most of the servers were at 2% to 5% utilization," he said. "In virtual settings, not only could we host multiple applications on a single server, we could build in a level of fault tolerance that wasn't there. That meant that we could be assured of a downtime measured in minutes, as opposed to hours or days."
"With every watt you save from a server standpoint, there's about another equal watt saved in cooling. As a result, we were able to change the way we cool the data center. We were able to remove or turn off CRAC units."
These energy savings have been all about following classic best practices. "We spent time clearing cables, avoiding barriers that inhibit air flow, setting up hot-aisle/cold-aisle arrangements, and ensuring we had proper placement of our CRACs. That's all what led to a green data center," Nichols said.
New horizons for a virtualized House: BC/DR, improved security
Creating this much room between the maximum power ceiling has opened up new possibilities for HIR to provide additional services to member offices.
"In many ways, the House is a collection of small businesses," said Nichols. "In the past, members have had physical servers in the office. We'd offer best practices for backup and maintenance." That member's office IT staff was responsible for everything, from networking to databases to Web development.
Now, power consumption reductions have led directly to the ability to virtualize members' offices. Every member office's data is uniquely encrypted to his virtual server, with a different encryption algorithm associated with each office. That unique identifier was a key step. "When you can get that hurdle cleared and we can say we can uniquely secure your data, then we can talk about centralizing," Nichols said.
That common security posture made the HIR chief information security officer happier, said Nichols. Decreased administration and capital costs pleased members too, he said, in terms of acquisition and repair. Nichols is happy about improved business continuity and disaster recovery capabilities, as centralized data is now replicated to secondary data center outside of the district
Blade servers save space, energy in this green data center
The HIR enterprise operations department was able to take two rows of individual servers and collapse them into a single blade frame rack. That meant moving from about 300 servers to about 30. That one blade frame consumes about 20,000 W, Nichols said. His department has now created virtual servers for more than 150 member offices so far, which saves the direct cost of the server itself as well as administration and 400 to 500 W per server in each office.
Nichols keeps a close eye on the potential for virtual machine sprawl, a common headache for modern data center operators. "We're under very tight constraints in terms of who has the rights to create a virtual server," he said. "We have restrictions and safeguards to limit that. In our initial foray into virtualization, there were tradeoffs between achieving the greatest consolidation ratio and achieving it in a secure fashion. We had to examine the business process behind each one of the servers. It's gotten better with third-party tools that can monitor traffic between hypervisors."
There are other possibilities under discussion now, like the potential for creating a private cloud or rolling out desktop virtualization down the road. Not every technology is going to be right for this environment, however. "If you can't clear the security hurdle for deployment, it's not going to be a good fit," said Nichols.
For now, Nichols maintains a heterogeneous virtualization environment and is keeping an eye out for other areas where the office can reduce energy consumption or improve utilization. "We want everyone competing, whether they're hardware or software vendors, so that we can get the best bang for the buck for the American taxpayer."