Problem solve Get help with specific problems with your technologies, process and projects.

Lessons from Katrina: Part 4, Resuming operations with end users

In part four of our series on rebuilding your data center after a disaster, Carrie Higbie has guidelines on how to get your users back into the network.

Editor's note: In part four of our series on rebuilding your data center after a disaster, Carrie Higbie has guidelines on how to get your users back into the network.


Here is where the rubber meets the road. With the necessary server, network and infrastructure-based services back on track, you will now need to add your users back to the network. This step can be the hardest. Permissions and user access will need to be in place in order for the end users to function. Training will certainly be a factor as new systems and operations may be in place that alter job functions and processes. Workforces may be entirely new or a combination of old and new, and passing knowledge of business operations and new security procedures will keep help desk calls to a manageable number.

After ordering hardware for the end users and working to install the necessary software, there will be packages that are used on a daily basis that may not be installed due to lack of knowledge of their existence. Vendor specific packages that have been installed for certain departments that occurred outside of the direction of IT are good examples. Custom forms and custom reports may need to be rewritten or revamped for newer revisions of software, especially if they originally resided on a desktop and were not backed up on a routine basis. These will identify themselves through the testing phases listed below.

Let the training begin

If your software inventories and serial numbers have been documented and exist outside of your primary business site, it will be much easier to obtain duplicates. If inventories do not exist, but your software was registered and licensed, the original vendor can help you determine what software a company owns. If registration did not occur for whatever reason, be prepared to order packages as they are identified during the testing. There is no guarantee that the same version will be available when new desktops are configured as software is routinely updated and support for versions already in an end of life cycle may no longer exist. Some testing will be needed to assure functionality between versions and should then be communicated to end users through the training phase.

Initial training will consist of new operating system procedures, network services such as logging in, new virus protection and security applications that my be new to the end users, as well as specific application training. For instance, old workstations may have been Windows 2000 and now running XP. Users will function more efficiently and effectively if they understand the ins and outs of the new operating system. Printing is also a good example. On the new systems, devices may have been renamed and parameters may have changed.

Read more: Lessons from Katrina

Part 1, Rebuilding a critical infrastructure

Part 2, Operating from a remote site

Part 3, Replacing active components

Users may see other applications that are foreign to them. Your help desk could be inundated with calls about what an end-user thinks a problem is based on something new they see on their system. This may not have anything to do with the problem at all, but since it appears as a change, the end-user may be blamed. This can lengthen problem resolution time if not addressed up front.

It is a good idea to install a help desk trouble ticket on each workstation. This should include information necessary to assist with troubleshooting. Include items such as all running applications, task being performed at the moment of error, the exact syntax of any error message, time of day, date, is the service running locally or remotely, etc. Based on your initial testing of the systems, it will be easier to tailor the questions to your business needs. As your support personnel may be new, you will want to assure that their input is coordinated into the form as skill sets may be different between people and specialties. Include enough information to assure quick resolution but do not make the form so difficult that you get a bunch of words on a form that lead to more confusion than resolution. Bear in mind that end-users may have no idea what some of the questions mean, so print screens and other visual triggers will help with your information gathering.

Haste makes waste

While it is desirable to return to normal operations as soon as possible and return the redundant site to its original purpose, a period of time should be allowed to work with the new services, applications and hardware to assure that these tasks can occur without malfunctions. Training teams will need to develop any training on new applications and procedures to be communicated to both new employees as well as those that have worked for a company for a period of time. This training should include new operating system procedures, security procedures and of course new operating procedures for any new version of an application that they will use in the course of their duties.

If you have a good process book, it is advisable to have the person originally in the position to work through the processes on the new system. This will provide insight into changes that can then be updated allowing all processes for the new systems to be part of your revised business continuity plan. With a disaster of this size, a company may find that their user community has moved, relocated or in some grim instances, may have perished. It may be advisable to set up remote services for the most critical applications to facilitate this process with the most educated personnel.

During this critical process, you will want to assure that the programmers are available for any custom applications and vendor representatives are available for COTS. Audit teams will be responsible for validating all results from the data input and documentation teams will be responsible for updating all business continuity plans. The network team will also be needed for any security issues that may arise.

If at all possible, the testing should occur over a 30-day period. This assures that all monthly operations are addressed including closing out accounting periods, order processing, invoicing, account payable runs and payroll processing. Where shorter timelines are necessary, the time frame for testing should include all critical tasks during the testing period. This will allow you to identify problems while you still have the luxury of working of your redundant site on what is likely to be older hardware and software versions than your new site.

Be prepared with a good help-desk tracking software that allows your help-desk personnel to search on existing problem/resolution sets to aid in speedy resolutions with other users within the same job functions. Logging will be essential in working with your hardware and software providers. The helpdesk reports will also facilitate updating your business continuity and disaster recovery plans post cutover to your primary site. If you are working with a largely new end-user base this will also assist with ordering software and other services needed to complete all processes that run a business.

The audit and training teams will be working closely with end users to verify data and new hire training manuals. The disaster recovery documentation and logging teams will be responsible for communicating changes to other users with similar functions and concurrently updating the disaster recover/business continuity manuals. Job descriptions may also change through this process. Some jobs may be eliminated due to automation and other new positions may be necessary either temporarily or on a permanent basis as duties are refined, audited and documented. Do not skimp on testing, auditing and documentation, as this could be a costly mistake. Be prepared to train new personnel and retrain existing personnel due to changes in functionality. Lastly, do not overlook the help desk and the knowledge gained through problem resolution that will define new business procedures moving forward.

Carrie Higbie has been involved in the computing and networking industries for nearly 20 years and has taught classes for Novell, Microsoft, and Cisco certifications as well as CAD/CAE, networking and programming on a collegiate level. Carrie currently works as the Network Applications Market Manager with The Siemon Company, where she provides liaison services to assure harmony between active electronics and networking infrastructures. She participates with the IEEE, TIA and other consortiums and works to further educate the end-user community on the importance of a quality infrastructure.

Dig Deeper on Enterprise data storage strategies

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.