Open source monitoring software should get a boost with the expected final release of the Assimilation Monitoring...
Project in late April.
Originally, the Assimilation Monitoring Project (AMP) was started to fix the configuration woes and workload issues present in traditional monitoring software, but has evolved to include automation and a unique brand of scalability. AMP founder and project leader Alan Robertson expands on the details of its final release in this Q&A.
How has the Assimilation Monitoring Project evolved since you started?
Alan Robertson: The project started out with one major emphasis and now it has two. When I started the project, I was initially intrigued by the idea that I could do monitoring without increasing the workload on the central system as servers were added -- that is, make monitoring scalable. But I realized that this was a problem only for a minority of potential users.
Once I saw that I needed network discovery to gain scalability, I realized we could do a lot more discovering and then showing you things, such as what services you have forgotten to monitor and dependencies between services -- and in fact, automatically keep more or less any set of facts about a data center's configuration up to date. It's a dirty little secret that people don't always follow procedures, and that they don't always know what all their servers are supposed to be doing or how they relate to each other. As a result, things like configuration management databases (CMDBs) range between imperfect and nonexistent.
Aside from stealth discovery, what are the advantages of the project?
Robertson: The project's scalability means you won't have to add servers or reconfigure your monitoring infrastructure as you grow. The fact that the discovery is integrated with monitoring means that the job of configuring the monitoring software is much easier. You don't have to tell it what to monitor; it can tell you. In future releases, we may even be able to completely automate some portions of monitoring configuration. We can also find gaps -- places where things that you're running aren't being monitored. This helps you improve the quality of the service you're providing.
You've mentioned scalability in the past, but can you describe specifically how assimilation monitoring is scalable?
Robertson: Nearly all of the work in monitoring is asking the variations on the question 'Are you OK?' and getting the answer 'Yes, I'm OK.' For conventional monitoring systems, this is all performed centrally, so as you add servers to be monitored, you have to grow your monitoring infrastructure because this poor central system gets busier and busier. With our system, this 'Are you OK? Yes, I'm OK' work never goes up as you increase the number of monitored servers.
The method we use for this is so simple, you could explain it to anyone. On Wednesday nights at my church, when we share a meal, people stand in a circle, hold hands and say a prayer. If someone falls ill while in the circle, the two people holding their hands on either side would cry out, 'Gramma Bee has passed out!' and someone would rush to help. If you look at this arrangement, you can see that the same structure works for three people, 30 people or 3,000 people. Everyone is just looking after their two neighbors -- and everyone is being looked after by two neighbors. We use a variation on this neighbor monitoring arrangement in our system. As a result, it scales in a radical way -- workloads for the 'everything's OK' case never goes up.
What are a few 'real world' scenarios that sysadmins could tackle from the project's interface?
Robertson: Although the method of monitoring I described is simple, it is radical when compared to traditional server monitoring tools, and integrating it with continuous stealth discovery makes it even more so. As a result, this first release is primarily a proof-of-concept release -- not intended for real production. We want to see how well received the concepts are, and how interested people are in using them.
With these caveats in mind, admins can see which servers are up; which are shut down; which crashed; how long they've been up or down; and a lot of detail about IP addresses, MAC [Media Access Control] address, switch cabling and similar information.
Release candidates 1 and 2 are up and kicking. How soon do you expect the final release of your open source monitoring software?
Robertson: Release candidate 4 is out the door. Unless something unexpected shows up, it will be the final release candidate for this first release. I expect it to be out by April 20.
Once that final release is live, what do you have in mind for the project down the road?
Robertson: I have a list of more than 150 items that deal with that question, but the most interesting ones relate to the discovery data. We have a rich tapestry of detailed and up-to-date knowledge about your data center that you probably don't know and certainly don't have in one place. How can we use this data to bring insight into the process of managing and planning your data center? How can we use our connectivity graph to show you visually how things relate to each other and what you need to know for all the jobs you have to do without bombarding you with unnecessary information?
There are also lots of mundane things and expected features to add, such as service monitoring, real alerting, configuration automation, lots more discovery modules, support for most modern OSes, high-availability configurations, network transactions, encryption, compression, secure communication. A little farther down the road are things like statistical monitoring, controlling the servers, integration with other data center systems, such as trouble ticketing, CMDB audits, etc.
This first release has enough features implemented to demonstrate the project's potential to make system administration easier and provides a solid base for managing the overall data center. What we need is for people to download it, join the mailing list, comment on it and contribute to it. Because it is an early release, the possibilities to make a difference to the project and the lives of system administrators are great. So, I'd like to encourage your readers to come along, give it a shot and let us know what they think.