What is mainframe performance management? In the past, neither mainframers nor non-IBM IT types would have had any difficulty in answering that question: mainframe performance management is that part of application and system management that focuses on performance, not RAS (reliability, availability, serviceability). However, they would have had great difficulty explaining to the CEO or CFO why he or she should worry about it. This is because performance management in general, and mainframe performance management in particular, has historically been an after-thought, a "nice to have" that always took a distant second place to making sure that the systems were up and running.
Note: The idea of performance management is a bit fuzzy to some users. Especially in the age of virtualization, performance management is not about optimizing the performance of each application; instead, it is about ensuring worst-case performance of all applications that meets a service-level agreement – say, a five-second response time for any end user at peak load.
How have things have changed?
First, there's a new meaning to performance management. In the business intelligence space, performance management that allows CEOs to monitor and improve business processes via dashboards is a hot topic -- but it's completely different from the old meaning. So, when you talk to the CFO now, he or she may know that performance management is important, but may completely misunderstand what type of performance management you are referring to.
The last four years have seen a major resurgence of mainframe spending, driven by the proposition that consolidating Unix/Linux workloads on a mainframe leads to significantly lower per-application total cost of ownership (TCO). But by focusing on TCO, mainframe server consolidation has shifted the relative importance of robustness and performance. That is, the better performance you get out of your applications, the more you can run in virtual machines on the mainframe, and the lower your per-application TCO. This, in turn, translates to less need to add capacity or new systems as user demands or the number of applications increases. To many users, mainframe robustness is a given; what really delivers added benefits is effective mainframe performance management.
So, mainframe performance management has become important. At the same time, the influx of Linux applications (and, secondarily, increased end-user Web access) has fundamentally changed performance management tasks for the mainframe. In the past, mainframe performance management was focused on managing performance inside the machine, with relatively little need to consider communication between virtual machines. Now, with Unix/Linux apps that often access each other via Enterprise Application Integration (EAI) or communicate outside the organization via the Web or Intranets, performance management must be "end-to-end." Network communication and client processing outside the mainframe and outside the data center must be optimized along with communication between virtual machines (rather than from machine to machine).
Here's another area in which the game is changing: software development. In the past, stress and volume testing were an afterthought in the development process, often curtailed or not done. By the same logic, few organizations attempt to feed bugs detected during software operation back to the testers or developers semi-automatically. But the new consolidated Linux apps on the mainframe are so robust and their virtual machines so well isolated from each other that some organizations are actually running their development/test environments as virtual machines on the same machine as business-critical operational apps. This makes it much easier to pass problem information from operations to testing; and since performance improvements yield a bigger bang for the buck these days, speeding performance tune-ups and bottleneck solving through testing and back into operational use pays off.
What do the changes mean for the mainframe today?
The new tasks and environments mean that mainframe performance management needs to change. For one thing, there are many more virtual machines to take into account; so it becomes much more important to offer fine-grained per-virtual-machine monitoring and measurement, rather than treating the whole mainframe as a black box. Also, cross-operating-environment management, in which the same performance management tool handles both local and non-local, Linux and z/OS environments, becomes more useful.
The net of all this change for users is that they need to look for the following new characteristics in a mainframe performance management tool:
- End-to-end administration, in which all distributed components of an application are monitored and assembled into a coherent performance picture.
- Experience in and customization for both mainframe and Linux administration -- preferably with a common user interface.
- "Deep" performance management, in which the tool can look at multiple layers of the app's software stack and figure out which part of the stack the problem is coming from. For example, some users are finding that server-side data access is slowing performance, others that the split between client and server has caused excessive "bursty" communications to slow down response time at peak load.
- A high degree of automation. In particular, the tool should allow automated handling of such performance problems as poor load balancing, the "query from hell", or inappropriate online backup times. Also, the tool should provide interfaces that Linux as well as mainframe administrators can use with little training.
Initial indications are that users can extend the effectiveness of mainframe performance management software with a few "best practices":
- Using zIIP processors, which improves a common (data-access) performance bottleneck.
- Increased focus on database performance management (neglect of this is often a source of high administrative "people costs") and on doing such often-scanted tasks as database reorganization (automatically, if possible).
- Creating more effective communication between operational performance management and software testing/development organizations.
- Fine-tuning the balance between remote/Web client and server by shifting code (e.g., presentation logic) between the two.
Mainframe performance management: Important now and in the future
If you think mainframe performance management is important now, wait until you see how important it will be two years from now. One key reason is the "graying and greening" of business-critical apps. These continue to reside primarily on the mainframe in most large IT organizations. As more "graying" mainframe administrators reach retirement age, administrative tools such as mainframe performance management will have to compensate, or else risks to the business from app failure will climb. Likewise, as energy savings from improved hardware and data-center design reach their limits, "green software" that uses the mainframe more efficiently will become more important -- and that's where mainframe performance management shines.
There's another possible reason to expect increasing mainframe-performance-management importance. As I noted in a recent article, it is likely that the mainframe will play a key role in a new kind of "hub" architecture, in which all platforms may play a "peer-to-peer" role, but in which the mainframe will attract the majority of applications for particular needs such as security, robustness and flexible capacity. In such an architecture, mainframe performance management will be critical not only to load balancing across the diverse environments, but also for determining which apps and app instances should be put on which platform, and how they should shift over time as end-user demands scale and shift. The net-net? The new Linux mainframe workloads have changed both Unix/Linux and mainframe performance management requirements. As a result, mainframe performance management is important now to IT and the business, and will only increase in importance over the next two years. The action item for users is to identify the best software from vendors such as IBM (e.g., Tivoli) and CA (e.g., Wily) and the best practices to implement highly effective mainframe performance management.
ABOUT THE AUTHOR: Wayne Kernochan is president of Infostructure Associates, an affiliate of Valley View Ventures.