SAN FRANCISCO -- Standing in the middle of downtown San Francisco at 8 a.m., I was trying to get into the 365 Main data center facility and not having much luck. Even the lobby was off-limits until I could come up with a government-issued identification (ID) of some kind.
I was scheduled to tour the facility with Miles Kelly, vice president of marketing and strategy at 365 Main, but the two guards remained skeptical. Miles wasn't in yet. They took my ID for safe keeping and put me in a corner to wait while they monitored me from behind the security terminal.
When Kelly did arrive shortly thereafter, he explained that even 365 Main employees get that kind of treatment. No one gets in without clearance and government-issued ID -- not customers, not 365 Main staff, not anyone.
We started our tour in the basement and would work our way up.
The 365 Main facility is located in an active seismic zone -- not exactly the sort of situation that would allow a data center manager to sleep easy at night. But the people who designed this building took radical steps to make the facility as earthquake proof as possible.
The building is built on top of a system of 86 base isolators. Base isolators are constructed of rubber bearings and steel plates. They are placed between concrete pilings driven into the bedrock and the structure of the building itself.
In the event of an earthquake, the ground moves, but the structure remains relatively stable. The building's entire infrastructure is actually hung from ceilings rather than under the floor -- it's all above the isolation plane. All cabling is run with nearly 2 feet extra slack to allow for movement. Piping is fitted with flexible joists.
According to Kelly, each base isolator costs around $30,000. The building is one of a handful in the state using the technology, and it is outfitted with a sensor that reports back to the California Department of Mines and Geology, which is monitoring the building's performance in the event of a disaster.
The foundation of the building is also specially equipped to handle the strain. It dates back to World War II when the Army Corps of Engineers used the property to manufacture tank turrets.
Juiced up N+1
The facility gets it's primary feed from Pacific Gas and Electric Co. (PG&E) -- 34 kV.
"It is the largest independent power feed in all of San Francisco, four times what the second largest building in San Fran's financial district requires -- the 52-story Bank of America building in downtown," Kelly said.
The load is served by six transformers at 3,750 kVA each and is distributed by forty-eight 480 v/3 Phase power distribution units.
In the event that the PG&E feed goes out, 365 Main has 10 diesel generators on the roof. In the basement, three in-ground tanks hold 60,000-gallons of diesel fuel. The tanks are hooked up to three recyclers -- units that move the fuel around. If diesel just sits, it begins to separate. Fuel cleaning rooms keep the diesel fresh and well mixed.
The diesel generators are able to power the facility for 72 hours at full load.
Walking into the first generator room, the roar is deafening. Each of the 2.1 MW Hitec Power Solutions generators is spinning a flywheel with enough force to jumpstart the engines within two seconds of a power outage.
The cooling system is equally mammoth:
In addition to redundant power, the facility has redundant fiber optic feeds from 30 network providers. According to Kelly, the diversity of carriers not only makes the network more resilient, it also eliminates network hopping. The building also features a wireless network in the event that all of the network cables are cut.
The raised floor
365 Main has 90,000 square feet of rentable raised floor space. The floor is 24-inches high and manufactured by Jessup, Md.-based Tate Access Floors. It features concrete cores and seismic pedestals with a structural capacity of 250 pounds per square foot.
This is where 365 Main pays the rent on its juiced up super-facility. Every feature of this building has been designed to keep those 90,000 square feet up and running. The company is in the business of renting data center real estate: powered, cooled, connected and secured. The customer is responsible for managing what is actually plugged in.
Most of 365 Main's server rooms are multitenant environments, with up to 25 customers in a 10,000 square foot room. Service level agreements (SLA) state that the room must be maintained at 72-78 degrees, so technicians at 365 have to know what customers are brining into the room.
"The challenge is laying out what users go where," said Jamie McGrath, vice president of customer deployment at 365 Main. "A lot of customers had environments in closets. So we actually coach them and educate them on how we maintain our environment."
How does 365 Main measure power and cooling needs on equipment? McGrath said sometimes he uses his own historical data, but a lot of times he relies on customers' measurements on the number of amps a machine draws on a circuit.
"We do try to base our discussions on experience," McGrath said. "The nameplates are usually exaggerated. Typically a server uses 60% of what the nameplate says it's going to draw."
McGrath said the power per square foot in 365 Main data centers has quadrupled in the past three years.
Construction on this facility began in 2000, back when 365 Main president Chris Dolan was the director of design and construction for the now defunct data center hosting company AboveNet. McGrath was his partner, and together they oversaw all aspects of design and construction for the $1.1 billion project.
Shortly after constructing this facility to host the Bay Area's dot-com build out, AboveNet folded and handed the keys over to its financiers, Rockwood Capital. Then in 2002 "when nobody cared about data centers," according to McGrath, 365 Main founders pitched Rockwood to buy the building and set up a modern hosting facility.
Now 365 Main owns the building that is host to several customers, including Charles Schwab & Co. and the Oakland Raiders.
Despite the awesome budget and horsepower of this facility, it doesn't quite meet the Uptime Institute's standard for full redundancy. 365 Main data centers don't have 36-inch raised floors. Also, a Tier-4 certified facility has to have redundant chilled water loops.
But that doesn't bother McGrath. "We follow the [Uptime] tiers because they've become a reference point. All of our facilities are Tier 3-plus.
"We don't claim Tier-4," McGrath said. "But the bottom line is that the facility is as modern and reliable as any building out there."
Let us know what you think about the story; e-mail: Matt Stansberry, Site Editor