In today's modern data centers, multiple servers, applications and devices co-exist in increasingly interconnected ways. In such an environment, unauthorized and undocumented change can wreak unintended havoc on IT systems. A tweak to a database here can bring down a document management system and BlackBerry infrastructure there. Introducing an upgrade to a server the Friday before financial statements are due can cripple the reporting...
system. Stamford, Conn.-based Gartner Inc. estimates that for mission-critical systems, 80% of downtime is due to human error and problems created by process, such as inadequate testing and unauthorized changes.
To prevent unauthorized changes from taking a toll on systems, IT managers have resorted to change management processes, along with approaches such as Excel spreadsheets or Visio diagrams, to track system configurations. Such decidedly low-tech approaches no longer suffice in today's increasingly complex data center environments, where servers and applications are factors in myriad systems and departments. Adding to this complexity are such developments as service-oriented architecture and virtualization -- technologies that essentially obscure the relationship and dependencies among servers, networks, applications and systems.
"The typical IT environment is very complex," said Chris Matney, consulting director of IT services at Enterprise Management Associates (EMA) in Milford, Mass. "To mitigate the risk of changes, companies either have to add head count to IT or operate IT more efficiently."CMDBs and the ABCs of change management
One way for IT to operate more efficiently is by deploying a configuration management database (CMDB), a repository that stores relevant information about all the configuration items (CIs) that comprise a system. Vendors have introduced CMDBs and related technologies such as inventory, discovery and application dependency tools to enable organizations to track and document changes. The goal is to prevent the system downtime that comes with poor change management processes.
Increasingly, organizations have turned to CMDBs as well as change management processes to mitigate risk and improve quality of service. In TechTarget's 2007 Data Center Purchasing Intentions survey, more than 25% of 398 respondents said they already have a CMDB, while nearly 25% said that they would evaluate the technology this year. About 6% of respondents said that they don't have a CMDB but would purchase one this year.Ideally, by including information about CIs such as hardware, software, services, documentation, and users and how all these forces relate within an environment, CMDBs enable data centers to understand how system changes affect both business services and end users downstream. "The premise behind a CMDB is to have a single source of truth for the accounting of IT components," said Eric Ho, client architect at Houston, Texas-based BMC Software Inc. This capability is increasingly important given the complex nature of many data centers, where business services are made up of multiple applications running on numerous hardware devices.
Unlike asset management systems that track IT assets from an investment and lifecycle perspective, organizations manage CIs in an effort to make better operational decisions regarding changes, problems and incidents. And while there's no standard definition for CMDBs, Gartner contends that the technology should at best include data integration and federation (tools that point to where the configuration data actually resides), application dependency mapping, visualization and reconciliation.The vendor landscape
Many vendors have responded to the upsurge of interest in CMDBs by offering their own versions of configuration management databases. (The concept of a central repository for configuration items isn't new; the specific term CMDB is one set forth in the IT Infrastructure Library, or ITIL.) Some have spruced up existing configuration management products, while other vendors have gained a foothold by acquiring other CMDB vendors. Among the top providers are BMC, CA, Hewlett-Packard Co. and IBM Tivoli. These vendors offer CMDBs in conjunction with IT service management software products.
Over the past few years, these vendors have acquired smaller CMDB and related-technology providers to complement their service desk, configuration management, and systems and performance management offerings. Other vendors in the space -- such as Managed Objects, Symantec Corp., and Tideway Systems -- offer CMDBs to support the service management precepts of ITIL.
Joe Kennedy, the vice president of IT at State Street Corp., a financial services firm in Boston, is a relative veteran of CMDBs. In 2003 he earned his CMDB implementation stripes deploying software from mValent Inc. "The evolution to n-tier applications [applications that are distributed among multiple computers] has caused a proliferation of servers and environments," Kennedy said. "Today a single application may run across up to a dozen servers. With so many servers to manage, the need for a CMDB, and access to configuration information, becomes vital."
For Kennedy, an out-of-the-box tool serves State Street's purposes just fine. The product from mValent "focuses on managing configuration in the application server stack," Kennedy said. "It's nonintrusive and agentless and doesn't require us to put a footprint on every machine we maintain." With the tool, State Street can identify discrepancies in its environment. "If there's something wrong with an environment -- one server is working and one is not -- mValent can pinpoint discrepancies between the configurations and instantly identify configuration items that are out of sync."
While an off-the-shelf configuration tool works well for State Street, for many organizations a single CMDB product is only part of an overall configuration management solution. Tideway Systems' application dependency mapping, for example, isn't a CMDB but can be a core part of a CMDB in that it provides essential relationship information among components. Symantec also offers an automated tool called Veritas Configuration Manager that feeds server and application configuration information into CMDBs. "Our typical customers spend from $1 million to $5 million over the lifetime of their projects," said EMA's Matney. "There are a lot of ancillary products for a CMDB that typically come into play."
According to Dennis Drogseth, vice president of EMA, there's no standard model for implementing a CMDB. Some organizations do fine with a standalone product, while others need a CMDB that's integrated with their help desk or ticketing systems from the same vendor. And still other organizations may need a standalone CMDB that is integrated with third-party software to bring together disparate sources of configuration information.
But no commercial tool can fix failing processses, of course. Without a certain level of IT maturity, a CMDB isn't particularly useful. Matney said that CMDBs are best deployed at the point where IT is ready to move forward with change management processes that enable proactive rather than reactive decisions. "Organizations that are ready for CMDBs are those that are focused on higher levels of analysis so that they can help the lines of business quickly get to new opportunities," he said. Typically, such organizations have a good grasp on how to monitor their environments and handle configuration basics; in order to become more proactive, these organizations have to hone their change management processes, and that's where a CMDB factors in, Matney said.CMDB drivers and benefits
Matney said that all of the consulting engagements he has been on within the past 18 months have concerned CMDBs. "There's a huge amount of interest in CMDBs," he said.
He also noted that most CMDB implementations are at large companies in financial services or other industries in which IT essentially defines the business. "If a trading or transaction system goes down, the business grinds to a halt," he said. With so much on the line, these organizations are CMDB early adopters.
Organizations that have implemented ITIL are also turning to CMDBs. For Harry Butler, support center manager at electronics supplier EFW Inc. in Fort Worth, Texas, the driving factor for implementing a CMDB was to improve change management processes." Configuration, change and release; these are the three areas where we need to make process improvements," he said. EFW has a CMDB from CA that the company uses to determine relationships between hardware, software and processes on a server and how those configuration items affect end users. Previously, EFW attempted to get a handle on such information with Visio diagrams, but the IT environment changed too quickly for that to be effective.
With a CMDB, EFW is now in a position to add and modify changes quickly, but only if those changes happen within the context of a structured change process. "The CMDB tool is great," said Butler, "but if you don't have the change management process in place, it's not going to fix anything. You have to sign up for the methodology that sits behind it." (For EFW, ITIL is the methodology behind its CMDB.) At EFW, no changes to configuration items can be made without going through a formal change process. Chief among the adjacent processes are ITIL's big three: problem, incident and, of course, change management. Determining which kind of CMDB implementation works best comes down to two related factors: what business benefits an organization is seeking to achieve and which processes are used to achieve these benefits.
At Linklaters LLP, a global law firm based in London, two primary drivers for implementing a CMDB were remote management and disaster recovery. The firm has four data centers: two in the U.K., one in Atlanta and one in Hong Kong. "The data centers are managed by third parties, so we can't go down the hall to check on servers," said Simon Gilhooly, the global head of technical systems. "We need to have a good sense of the components of our critical systems such as document management."
Linklaters invested time and money building a disaster recovery (DR) solution for its data centers and wanted to make sure that the disaster recovery plan doesn't become stale over the course of frequent upgrades and regular maintenance. "We use the CMDB to document components as part of the DR change request," Gilhooly said.
To build a comprehensive CMDB, Linklaters used its existing incident and change management technology from BMC and integrated these components with BMC's Atrium CMDB. The firm chose software from Tideway to provide discovery and build an application dependency model based on Linklaters' infrastructure. In essence, the model automatically feeds a CMDB with information about IT and application infrastructure, calculating the dependencies so that the firm can accurately gauge the impact of system changes.
As for solid ROI for CMDBs, it's often difficult for organizations to quantify the cost of service improvements or even calculate how less downtime pays off. But data centers can achieve significant operational benefits. Dave Johnson, formerly the market segment manager at Symantec Corp. (Symantec recently acquired CMDB maker Altiris), said that one of the central challenges for data centers is configuration consistency. "Over time, the configuration of any given server drifts away from the norm," he said. "With a CMDB -- backed by change processes -- a data center has the ability to create standardized configurations."
At Linklaters, one of the benefits of a CMDB is using inventory information to manage hardware maintenance in outsourcing deals, Gilhooly said. "We're not wasting money on maintaining servers that we are no longer using," he said.CMDB caveats
Still, Gilhooly said that there are pitfalls in using a CMDB; a central one is the misguided effort to collect every bit of configuration information in order to justify the technology's existence. "It's tempting to gather lots of information as a way of selling what you're doing," he said. "It's a challenge to narrow our focus and be selective about what information we put into the CMDB." Trying to do too much with a CMDB -- gathering too many attributes about too many CIs -- is a common reason for CMDB failure. Collecting and populating a CMDB with data is an intensive exercise, and from a pure CPU processing perspective, querying the subsequent information in a CMDB can be unwieldy. At Linklaters, only configuration information that is relevant to disaster recovery is included in the CMDB. Other information, such as the available disk space for servers, just isn't appropriate, Gilhooly said.
For large environments, consultants recommend taking a fairly shallow approach to populating a CMDB. Configuration items may include server host name, the name of applications and their dependencies. More detailed information can be included in a federated model that can be organized by domains. A CMDB, for example, won't provide information about every router and host bus adapter, but it will point to a networking-centric database where that information can be found.
At EFW, Butler first tried to collect a broad swath of data through an automated discovery tool but soon abandoned that approach in favor of a "500,000-foot view" that encompassed configuration information about the company's mission-critical systems such as enterprise resource planning and email. Over time, EFW has expanded configuration items to include other systems such as its service catalog.
Johnson, the former Symantec manager, recommends that organizations take a similar, phased approach to implementing a CMDB. "You have to identify the problems that you are trying to solve first," he said. "When collecting inventory data, for example, you want to configure it with only those attributes -- such as connectivity to other systems -- that you want to store." Arlen Beylerian, vice president of product management for CA's business service optimization unit, has more quantifiable advice. "Organizations should aim for putting only 10% to 20% of attributes into a CMDB," he said. Additional configuration information can be stored outside a CMDB in a federated model.
Frequently the most important hurdle in successful CMDB implementation isn't technical or even project-scope issues. Consultants, vendors and practitioners agree that the most serious impediments to CMDBs are cultural and political issues. "The reason CMDB-type initiatives have taken so long to evolve is that there's nervousness around automating IT," Kennedy said. "Some managers would suffer with an inefficient system rather than give back resources."
A data center manager from a medical technology company who did not want to be identified said that at large companies, politics are especially problematic. His company is implementing a CMDB to get a handle on its IT portfolio; his company has plenty of lists of its hardware and software assets, but it's largely tribal knowledge that links assets together into a cohesive picture.
Since State Street has a global presence, Kennedy said that there is an awful lot of nuance involved. "It's a challenge just to define and understand the scope," he said. "You need to figure out how to engage the right stakeholders to collect much of the data that you need." For this data center manager, communicating the reason and value of a CMDB has been critical to overcoming resistance. "We have to clearly convey the value and how they can participate in it. We're trying to help ourselves help the business by understanding where costs are incurred." With the CMDB, for example, the company aims to improve its entire system lifecycle management process by looking at system utilization and retiring or upgrading systems that are gathering dust.
Travis Greene, chief service management strategist at NetIQ, a provider of systems and security management solutions based in Houston, Texas, asserts that process is paramount. "Without a strong focus on process, the CMDB success rate is zero," he said.
Despite the numerous challenges, EMA's Matney says CMDBs are a critical tool that enable data centers to operate more efficiently. By introducing automation to the process of managing configuration items, IT organizations can essentially keep tabs on the ever-expanding universe of configuration information with fewer resources. "Within the next five years," Matney said, "there won't be a Fortune 1000 company that can be competitive unless it has a CMDB."
Let us know what you think about the story; email Megan Santosus, Features Writer.
Dig deeper on Configuration and change management tools