Splunk gives Motorola Google-like insight into IT assets |
 |
By Megan Santosus, Features Writer
11 Mar 2008 | SearchDataCenter.com |
|
|
Mike Danley would be the first person to tell you that selecting a systems management tool at a large company can be a convoluted process. Danley is the IT director for e-business technology management at Motorola Inc.., the $37 billion communications company. In his job, Danley is charged with integrating the various components (the underlying technology stacks) of Motorola's shared-services environment throughout the company's supply chain.
But while Danley's group has had to troubleshoot problems within this distributed infrastructure, he never considered implementing an enterprise systems management tool along the lines of Tivoli . "Within the walls of a large IT world, there are dozens and dozens of management tools," Danley said. Going for another enterprise management system to fulfill his needs wouldn't have been practical or timely.
What Danley wanted was a tool to provide him with a view of all the various IT assets that run the relevant e-business components. So Danley turned to Splunk Inc., a provider of an eponymous open source tool that indexes IT-generated data such as logs, configurations, alerts, scripts and performance metrics such as power consumption.
|
|
|
|
|
Troubleshooting now involves looking at cause and effect and not just at one isolated issue.
Michael Baum,
CEO, Splunk Inc.
|
|
|
|
|
|
|
|
|
|
"The EAI [enterprise application integration] infrastructure is comprised of many physical systems and many technologies on many boxes," Danley explained. The middleware stack alone runs across about a dozen nodes.
Needless to say, whenever problems occurred, tracking down the root cause was a challenging exercise, as was determining ownership once the problem was detected. Resolving a help desk ticket regarding an integration problem might require a systems administrator to manually grep -- that is, write a command to search for a particular character string -- 75 files or more.
Here's how the process typically worked. Let's say a user has difficulty reconciling purchase orders received with those processed and opens a help desk ticket. The systems administrator then begins to gather logs and records from the various integration components, then searches them for any references to the purchase order number among the numerous files and systems involved in the transaction. Ultimately, resolving the problem can take two hours or more and involve multiple systems administrators and departments.
But with Splunk in place, a systems administrator simply plugs in the purchase order number in question into Web-based Splunk interface, which runs a canned search script to see where among servers the transaction got stuck; the problem is resolved immediately.
The index-and-search approach to IT data
Splunk co-founder and CEO Michael Baum likens his company's product to Google. Rather than search and index the Web, however, Splunk searches and indexes the data generated by servers, applications and network devices. As IT environments have become increasingly complex, Baum posits that the ability to search for relevant data is more useful than poring through the reams of reports IT administrators can get out of traditional systems management tools.
"IT systems have become so complicated that we have to get more sophisticated in deciphering the relationships within them," Baum said. "Troubleshooting now involves looking at cause and effect and not just at one isolated issue."
David Williams, the vice president of research at Gartner Inc. says that Splunk fills a gap not found in proprietary log management tools. "There are plenty of log file management systems out there, but most of them focus on logs from a security and compliance perspective," he said. They search log files looking for discrepancies that could suggest a security breach or compliance problem. "Splunk is really designed for IT operations in that it collects huge amounts of miscellaneous log file data from disparate sources and makes sense of it."
ITIL and free downloads seal the deal
As an IT Infrastructure Library (ITIL) shop, Motorola's help desk group's functions are structured based on ITIL precepts including incident management, problem management, availability management and service-level management. Danley was initially drawn to Splunk in January 2007 as a way to tackle those processes.
Splunk is free to download, which didn't hurt either. "The thing Splunk had going for it was a free evaluation. I got a limited usage with the download, and I could immediately evaluate the tool's potential," Danley said.
Once Danley had a look at the tool, he opted to buy it. For the initial setup, a Splunk representative worked on-site with Danley's team to index the log file information thrughout the relevant architecture. "It was real easy to config," Danley said.
Streamlining problem management
Currently Danley uses Splunk for the EAI infrastructure components found in 12 servers -- and collects 6,000 data points or sources. (A separate IT security group within Motorola uses Splunk proactively monitor the network by indexing data from network devices, firewalls and routers.) Additional component layers Danley hopes to load into Splunk include B2B, EDI and LDAP among others.
While Splunk has been in production only a short time, it has improved the process of troubleshooting considerably. Systems administrators no longer have to search log files looking for a problem's origin. With Splunk's indexing and searching capabilities, the relevant information is essentially delivered directly to them. As Danley sees it, Splunk is not so much a systems management tool as an aggregator that provides him with a searchable view throughout his e-business infrastructure.
"I get my own little picture of all my IT data," he said.
Let us know what you think about the story; email: Megan Santosus, Features Writer
.
|
|
|
|
