News Stay informed about the latest enterprise technology news and product updates.

Colocation providers reflect on robbery at CI Host

CI Host in Chicago has been burgled four times in two years, which has prompted other colocation providers to question their own security.

Chicago-based CI Host is a legitimate company, providing more than 250,000 consumers and small and medium-sized business in 190 countries with managed Web hosting, dedicated server and colocation services. So how is it possible that the facility has been robbed four times in the past two years?

According to reports, CI Host's night manager was attacked last week by intruders and assaulted with a Taser and blunt object. The perpetrators then stole at least 20 servers belonging to CI Host and its customers.

This event took place despite the company's Web site pledge to customers of its Family Colocation service: "Your machine will be housed inside a secured shared colocation area."

I told CI Host I was coming to pick up my servers. That's when I found out my servers were stolen.
Nick Krapf,

According to a published report, CI Host chief corporate counsel James Eckels hinted that the robbery might have been an inside job, saying, "The thieves were likely familiar with the building layout, the company's operations and the technology involved."

Statistics from Migration Solutions, a data center consultancy, suggest that the possibility is quite likely. Migration Solutions estimates that acts of theft, fraud and vandalism in the data center are three times more likely to be the result of an inside job than to be the work of an outsider. And about 65% of data center security breaches and other incidents are driven by malicious intent rather than economic gain, executed by disgruntled current or ex-employees, according to Migration Solutions.

Several angry CI Host customers have discussed the possibility of filing a lawsuit against the colocation provider for its negligence and failure to communicate the theft until days after it happened.

Nick Krapf, president of the gaming network site, said the incident in Chicago cost him $15,000 in servers and a damaging hit to his customer base, which didn't have service for at least three days. But the worst part was the company's failure to communicate, he said. "At first, we were told the servers went down due to a power issue. ... I told CI Host I was coming to pick up my servers. That's when I found out my servers were stolen."

Security lessons for users and providers
At press time, CI Host had not responded to questions about how the security breach occurred and how it would compensate customers, but other colocation providers had plenty to say.

According to Chris Crosby, senior vice president at Digital Realty Trust, "Security is a paramount issue for customers with installations in colocation facilities. It is overwhelmingly the most important thing they are seeking in a facility with 80% of customers ranking it No. 1."

Knowing this, Digital Realty uses a multilayer security protocol to protect all its facilities. A four-level access control system is the foundation of the system, limiting access to the facility to authorized people. The facilities also have a check-in system that tracks everyone who is in the facility and limits the areas that they are approved to be in. There are also biometric access points to equipment areas where customer installations and other critical systems are housed, he said.

Similarly, the Planet, a Houston-based company that owns and operates six data centers containing more than 40,000 servers, said it that has instituted strict security procedures. "Any time people come in and out of our facility -- bringing equipment in or out -- they have to go through the multiple points of security every time," said Yvonne Donaldson director, public relations at the Planet.

"Customers should expect this kind of access control system in any facility they are affiliated with," Crosby said.

Unfortunately, many data center facilities make a show of security but don't really stand up to serious scrutiny, said Chuck Goolsbee, blogger and vice president of Tech Ops at Seattle-based colocation facility digital.forest. "The 'rent-a-cop' types that they hire to work there are not really qualified to act as security gatekeepers. Minimum wage … and complete ignorance with regards to the equipment they are charged with guarding is what I've seen, at major players from Exodus (RIP) to InterNAP."

When worst comes to worst
Obviously CI Host should have had certain controls in place to mitigate its security risk, but the reality is that it's quite difficult to create a break-in-proof facility, said Aaron Sawchuk, co-founder of the Massachusetts-based ColoSpace.

"This event certainly has encouraged us to re-examine the physical security at all of our sites. We review these practices on a regular basis anyway, but we will be paying special attention to things like common hallways other access areas," Sawchuk said.

Even so, very few colocation providers should be patting themselves on the back, Goolsbee said. "So long as facilities are unmanned, this will happen. The concepts of a 'lights out' facility and a 'secure facility' are in so many ways mutually exclusive."

When a security breach does occur, colocation providers should regard CI Host's response as an example of what not to do: that is, lie to customers about the source of the downtime.

"They blamed a router issue," Goolsbee said. "What good does that do anyone? Customer equipment was gone. I can't imagine them maintaining any credibility in the marketplace after this has come to light."

Sawchuk agreed that notification and crisis management could have been handled better. There is never a good way to "spin" data center security problems, but the period of misinformation definitely hurt the firm more than it could have helped, he said.

"The Internet message boards and email lists are rife with examples of pissed-off customers who were led on for days thinking their servers were just down rather than stolen," Sawchuk said. "At the very least, that lack of information prevented the affected firms from notifying banks of possible credit card theft, and other important regulatory requirements. This piece of the event in and of itself could lead to a major legal headache for CI Host."

Let us know what you think about the story; email Bridget Botelho, News Writer.

Also, check out our news blog at

Dig Deeper on Data center design and facilities

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.