A lot of companies got caught flat-footed in the late nineties when the world-wide web went big. Ironically, some of these were well-established enterprises who understood the value of technology and had extensive computing resources deeply integrated into their business processes.
Supporting the web also meant changing well-established technical support structures. Before the 90's, most companies used on-call rotations where the on-call person carried a pager in case of emergencies and had up to fifteen minutes to respond. After that came cell phones and dial-in access from home.
However, as the web grew in importance companies realized problems had to be fixed immediately. Fixing problems immediately meant a 24x7 on-site technical presence. There are several strategies for dealing with these requirements.
Brute force: 24x7 systems monitoring
This approach takes the existing staff and mandates on-site coverage during the week with round-the-clock shifts of varying length. The on-site coverage desk is placed in a different room, far away from the regular work area and daily activities so the designee can spend all his or her time monitoring systems without distraction.
The chief advantage to this scheme is that it is quick and cheap. It can be implemented as soon as everyone is ready for extended shifts with existing staff. But, technical support people tend to be grouped into small teams with potentially narrow specialties. To prevent burnout the small teams must be combined into larger teams for the support cycle. If too many teams are stuck together on one rotation, the on-site person may need to contact a subject matter expert (SME), no matter how good the problem determination documentation, if the problem is outside of his or her experience thereby extending the potential outage.
On the resource management side, this approach tends to wear people out faster and creates a certain amount of ill-will amongst the technicians. We might also assume that a highly paid technician should have more productive things to do rather than hit enter on a monitoring screen.
Technical support outsourcing
Outsourcing technical support is another scheme. It means outsourcing off-hour support to another company while primary support remains in-house. There are a lot of advantages to this approach over brute force.
First, the provider manages the manpower issues inherent in undesirable late night and early morning shifts while the company staff keeps a normal work day. Second, an outsource provider can usually provide talent more cheaply than a client could hire it. Third, the employee technical support staff is free to spend its days concentrating on maintenance, administration and optimizing system performance free from the distraction of nagging problems.
However, there are a couple of issues that must be considered. As in any outsourcing agreement, the datacenter owner loses a degree of control. If something happens that the business doesn't like it is easier to discipline an employee than negotiate with a contractor. Outsourcing also creates a certain level of insecurity with the hired staff that doesn't go away despite management's repeated reassurances. It's also possible, as time goes on, problems happen and new systems are implemented, the outsource provider may end up knowing more about the systems than the technical support staff.
Perhaps the most important issue is third party access to company data and hardware. Data confidentiality is a very big issue for insurance and financial companies who must go to great lengths to limit contractors' access. Even the normal type of access operators need, such as system consoles and hardware must be carefully administered. And these problems are even worse if the outsource provider is overseas.
This idea is a more reasonable approach to the brute force method. The heading says "super operator," but it could also be as simple as finding a few systems programmers who don't mind working at night.
Instead of putting the entire technical support staff on shift work, some companies may prefer to offer a premium or bonus to volunteers for working at odd hours. The business must also supply enough education to turn specialists into generalists with broad knowledge of the systems and the applications running on them. In the end the goal is to for these personnel solve at least 95% of the problems before calling someone else.
The chief disadvantage here is it takes longer. Not only do you have to wait for volunteers, there is a large investment in time and money for training. Scheduling vacation may also become a problem depending on the depth of the support team. However, the advantages are numerous. The people working at night are there because they want to be and the expertise garnered through working on problems stays in the company. In the long run, this option is also cheaper than outsourcing and doesn't have data confidentiality issues.
Ten years ago, if CICS went down, a department in the company couldn't do its work for thirty minutes. Now an ailing system can mean hundreds, if not thousands, of customers all over the world don't have access to their money. Change is never easy, but I'm sure all of us use the web often enough to understand the frustration of a dead link or the occasional indeterminate money transfer. Understanding the frustrations teaches us the importance of vigilance supporting our systems. With that in mind the next question is how.
ABOUT THE AUTHOR: Robert Crawford has been a CICS systems programmer off and on for 24 years. He is experienced in debugging and tuning applications and has written in COBOL, Assembler and C++ using VSAM, DLI and DB2.