This is part two of the workload automation podcast with Robert Stinnett, a data center automation architect with CARFAX, Inc. You can find part one here.
One of the things we've been reading is about DevOps and Agile and how some people seem to resist it. Do you think there is a bit of foot-dragging that goes on with the workload automation evolution? People are concerned that they're going to lose their jobs. That's a very real fear with the current economy – lots of people are looking for work. Have you seen anything like that in your experience?
Robert Stinnett: What I've seen is a culture shift. No matter what you call it, Agile was a culture shift when it first came into the development world, and DevOps is a culture shift when it comes into the operations and development world. The fear is of the unknown. Most people in IT are Oracle experts, Puppet Masters or the Java gurus, and now we start saying, "Hey we want you to start working on these DevOps teams -- get involved a little with the network and get involved with the database."
Sometimes IT guys think "Wow, that's going to water my skills down," or they do view it as "Hey, I'm the guru here and now we want this to be a team project? Is this going to eliminate me or reduce my position?" Both those questions, I think, deal with false assumptions that people make. Even though we do have a rough economy, there are signs of things getting better slowly. Out here in the Midwest, for every person in IT out here, there's three jobs waiting for them. We have near zero unemployment in the IT field because every company, every business, knows that IT and technology is critical to the success of their business.
It's a little misleading sometimes when you look at the economic data and try to apply it to technology fields because there's a big difference. I think that as more and more people realize this, they see more companies and coworkers adopting a DevOps system. They realize the thoughts they had of perhaps losing or being put out of a job because of automation or a new philosophy of work are really not true because it enhances their skills, first of all. For example, you have a Java guru who becomes knowledgeable on how Oracle works. Now that Java guru, in the eyes of the company, and of many other organizations out there, is even more valuable, because not only does he know Java, he also can make his way around an Oracle database. Now, by no reason is he an Oracle expert, but as we all know, the time is long gone where anyone in IT is doing a single job.
You really have to be multitalented in IT. You have to understand not only how your stuff works but how the other systems work; at least have what I call “talking knowledge” of that. I may not know exactly how Oracle works and I never want to know that, but I can hold a conversation to help you understand the requirements, and understand how I need to interact with your system. Plus, I can understand what you're telling me – that is very important.
The whole DevOps thing is bringing all these people together. So before, we worked in silos. We had our development silo, operation silo, Oracle silo and network silo, so we said "Hey, let's bring all of you guys together." Now, we still need our network experts; we still need the Oracle experts and the Java experts, but we're not bringing you together so you all can discuss this and work on it together.
It does not mean that the Java guy is going to do the same job as the lady who does Oracle. What it does mean, however, is that when questions come up or roundtable discussions on how to make a decision, you're all making decisions with the input of everyone that is going to be impacted. You are not making these decisions in silos anymore. I really think the DevOps movement is great because it brings amazing efficiencies to the whole IT process. I worked on projects in my career where we went through an entire development cycle then at the very end the network guys said "Well, that's not going to work," and you have to go back almost to square one.
Getting everyone together helps eliminate that and I think it just makes common sense. If you go to many companies' IT department, you're going to see something really interesting. In many organizations, the cubicle walls are gone! I see more and more open floor plans, which helps DevOps by taking down the walls and the barriers. We want people talking, because the more people talk the better things come out in the end. Who wants to sit in meetings all day long where you take notes and find out that you missed some things, so you have to have another meeting next week and a meeting to follow-up that meeting? Why not just turn your chair around and talk to the person right behind you, have a conversation others can hear and let them join in with their opinions? That is really the fascinating part of DevOps for me.
I think the DevOps movement is really going to help the workload automation evolution, too. Before, automation was really an afterthought. Developers would and get the code base up and running, the database people got their part up, the network people get their part up, etc. and then at the very end, someone remembered, "Oh yeah, we've got to bring the system up and down automatically. We need this data transfer to happen. Let's see, how can we do that?"
Usually, one of several things happened. They'd either write an automation themselves and end up with 500 different home-grown scheduling systems running in the environment, they just ignored it and said "Well this is just the way it is, somebody has to go push the big green button every day to make this happen," which of course is inefficient, or they go to "throw it over the wall" and find that there was nobody on the other side to catch it. You had a lot of projects that sort of fell into abandonment as soon as they went live because no one really wanted to claim ownership for the thing.
I think the losses are helping drive IT folks to do the automation. They involve workload automation and datacenter automation from the get-go. As they go through the process of development, they say "Hey, we can jump in at this point and see what we can do to help make this a more efficient effective process." DevOps is great for automation and I think the two go hand in hand. I hope, as time goes on, to see more companies embracing this philosophy. I think it can do nothing but good.
I went to the Computer Measurement Group conference last December and met somebody from PayPal, Inc. As we were having lunch and we got to talking about what I did here at Carfax, I said "Oh you know, I do a lot of the workload automation, volume control, Control M…" and she said "Oh did you know that Control M is what runs our reconciliation process in the back end for our debit cards?" I was shocked!
This is a great example of how workload automation works, because that's not typically a batch process but an event-driven process. You have to break that mindset of time-based processes and look to more event- and resource-based. I think PayPal is a great example because they don't know how many transactions they are going to do per day on their debit cards, but as those events happen you can't process all of them at once. If I go have a million swipes to that card, your systems are probably trying to process and reconcile in a five-minute time frame. By doing effective resource and workload management, IT folks can look at the automation and say "Okay, I can process this in this window, this in the next window. … Oh, now there's something else happening here so it slowed down."
Like the old days, PayPal still seems to do this reconciliation process in the evening or at night. I can tell by how the emails come in. I get an email for each reconciliation when I use my debit card and at this time I'll get three in a minute and maybe a half hour later I'll get another two and, finally, in a another half hour I'll get one more.
Even though you don't know what's going on behind the scenes, knowing they do use automation tools and knowing the overview of the process, you realize they're processing all these transactions in small chunks, not trying to process everything at the same time. When taken in small chunks, the resources become available on their end as they process and, of course, the customer gets the information on the other side. It's pretty fascinating. I'm probably looking too deep into that process, but to me it's fascinating. It is workload automation in action.
If you are not careful, you can suddenly have 200 different home-grown systems running across the whole company. As the company grows, this is unsustainable because nobody really knows what everyone else is doing. That's usually at the point where companies say, "This is working but how can we make sure it continues working as we grow?" That's when they start looking vendors and trying to figure out "Hey, where can we put this? Can we put this under a package that will do all this in a centralized way?
Fourteen years ago when I was working for a large insurance company in the Midwest, if I had some data about agents, I kept saying, "There has got to be a better way." I wrote a little mainframe script and finally somebody said "Hey, if you enjoy doing all that it just so happens we have an opening in our scheduling department. How would you like to that?" I said "You know this sounds like a good, fun challenge. I'll try it for a while." So I tried it and I enjoyed it. And I enjoyed it so much I started spreading the word as it was and now, every day is fun.
Most people that work in any type of workload automation role or DevOps shop get to interact with more than just the IT folks. So when you're talking about workload automation, you have to interact with the business, because the business sets the priorities.
Take web auction store eBay, for example. The priority of eBay is not to see how many servers they can bring online or how much Java they can write. I'm pretty sure the priority of eBay is to see how many sales and listings they can get on a day to day basis. Of course, IT supports that, but IT is not eBay's primary role. You'd be hard-pressed to get any organization to say "Yeah, IT is our primary role." Now if you're a consulting firm or something, maybe, but even then these tools and technologies enable the business to work smarter, better.
One of the great examples is from the 1980s, when Kmart was the number one retailer in America. There were Kmart stores on every street corner. Now, most people they don't even know what Kmart is, or they say, "I haven't been to a K-mart in 10 years!" Walmart is the new king of the block – there's a Walmart on every street corner and mountain top.
There was a fascinating book about the fall of Kmart. One of the chapters in the book highlighted one of the reasons Walmart was able to pull ahead. At the time, Walmart was a tiny little Arkansas retailer that wasn't much. Here was the Kmart behemoth of 3,000-plus stores. What Walmart did that Kmart didn't back in the 1980s was invest in technology and in automation.
They got their warehouses automated. They got their stock replenishment systems automated. They knew what their customers were buying and they could make sure they tailored to each store. Kmart said, "Eh, we don't need that. We're the king of the hill." History now shows us what happened, and I think that's a great example there because no matter what you do, if you're selling wool sweaters, there's probably 10 different firms that also sell wool sweaters.
So how can you differentiate? Well, you can differentiate through your pricing or through your corporate philosophy. But behind the scenes, at the end of the day, it really is about how good your IT is. That is what drives a lot of companies because you have to be able to react quickly to changes – do things better, more effectively, more efficiently.
At the end of the day we're all selling wool sweaters and if there is a company that can do it better it’s because they can restock those sweaters faster. They can keep the costs down because they know their trucks are going up to New York, so they know they're going to pass by three other stores they can stop off along the way and save them three truck trips – that's what matters. Again, I think that's what is driving a lot of this whole data center automation, which encompasses workload automation, because companies are realizing, "I can make my company better if I can get my IT processing and processes more effective and more efficient."
I guarantee 20 years from now we'll look back and there will be somebody that overtakes Walmart. There is always someone to say "How can I do it better?" Again, this goes back to agile: Doing things in small incremental steps, but each step gets better along the way.
What piece of advice would you give to IT shops working towards cloud-based process automation?
Stinnett: The best advice I can give them is before you try to jump to the cloud make sure you have your house in order in terms of your internal IT resources and your internal IT staff. Jumping to the cloud with faulty or incomplete processes makes the problem even bigger. So my best advice is to make sure you don't try to run into the cloud until you're ready and until you have a good reason to be there.
If you can handle all your IT stuff internally, there might not even be a reason for you go to the cloud. We get caught up in this cloud buzzword – everything has to be in the cloud. It's great. It has its benefits, and it works well for many things, but sometimes you don't need it. Just like some organizations don't need Oracle; MySQL works just as well for them, you have to need the cloud and have a business reason for going there – a justifiable reason.
Sometimes it is cheaper just to go buy another server and bring it online internally than to try to get IT in the cloud. Of course there could be security and other reasons why you have to make sure you understand cloud before you go jumping in as well. Again, make sure you know why you're going there, make sure you have a plan to get there, and make sure that internally you have things in order first – so you're not just moving the problem from one location to another.
ABOUT THE CONTRIBUTOR: Robert Stinnett is a data center automation architect who works for CARFAX, Inc. Stinnett has spent more than 20 years in both data center and workload automation, has experience in both the distributed and mainframe side of data center automation, and has written several white papers on data center automation and optimization. Stinnett has also been a speaker at various conferences on the subject of enterprise automation and has written a number of public domain utilities to assist others in realizing the benefits of end-to-end automation efforts.