INFRASTRUCTURE MANAGEMENT TIPS
How to prepare for remote data center maintenance trips
Kyle Rankin, Contributor
11.03.2008
Rating: --- (out of 5)
|
I'm definitely a fan of remote management of servers, but there are times when you just need to be at a remote data center. Whether during an initial build-out, or when you need to perform major upgrades, some tasks require a physical presence. When your data center visits are as seldom as mine, planning becomes very important -- even more so when your data center is out of state or out of the country.
Remote data center visits are almost always on a limited schedule and as such they succeed or fail based on good organization, planning, experienced administrators and a little luck. I've certainly been on the receiving end of data center trips gone bad, and while this article can't provide experienced administrators or luck, here are a few tips to help ensure your next remote data center trip is a success.
Plan and time every task
Usually when you plan a trip to a remote data center, there are specific tasks you are scheduled to perform. When you plan the length of your trip you need accurate timing for each task on your list. Break each major goal into smaller tasks and estimate realistic times for each based on the average time it should take, not the fastest time. If your trip is also being coordinated with engineers from third parties (e.g., to install new gear), be sure that you have accurate timelines from them for each of the tasks. If you plan to do your own work while other engineers are there, be sure they won't need much of your attention and assistance.
Over-schedule time
No matter how tight and accurate your schedules are, there is something about traveling to a remote data center that causes the unexpected to happen. As a rule of thumb, I like to budget in at least one empty day for any remote data center visit. That way, when the unexpected happens (and it always does) and takes me away from my main tasks, I still have one extra day to get everything done. While it does add to the budget of the trip, compared to the co
To continue reading for free, register below or login
To read more you must become a member of SearchDataCenter.com
sts associated with rescheduling flights or flying back to complete a project, one buffer day will more than make up for its cost the first time you have an emergency. As a plus, if you do actually get all of your work done on schedule, you might even be able to get a little enjoyment out of your trip.
Plan your tools and inventory
Nothing is worse than arriving at your data center to find that you didn't bring the right screwdriver, your fiber cables are two feet too short, or your rail kits are incompatible with your rails. If you only visit a data center once a year, you might not remember that it doesn't have its own label maker, rail screws, or cable ties. Plan ahead on what tools and gear each task requires, and in the case of cables, it's better to be three feet too long than three inches too short. It never hurts to bring extra cables, screws, and other gear. If you don't use it, you can always leave it behind for the next systems administrator. In addition, research which stores in the area (if any) stock data center equipment, just in case. Finally, if you had gear shipped to the data center separately, especially if it is from a third-party vendor, be absolutely certain all of the gear will arrive before you do.
Work ahead
I know that different administrators approach remote data center trips differently, but I've always been an advocate for working ahead. If I have two days worth of work and three days on site, I always try to get as much as possible done on the first day. In the best case I have a much lighter load the next two days, and in case everything goes wrong I not only find out sooner, but I have more time for contingency plans.
Have contingency plans
As the old saying goes, hope for the best and plan for the worst. What if your networking cables are too short? What if your rail kit doesn't fit your racks? What if your PDUs can't handle any more equipment or you simply run out of power outlets? Even with the best plans a project has a tendency to throw you a curve ball when you least expect it. I've already mentioned one of the more important contingency plans -- always know the local stores that stock CAT5 cables, rail screws, and other data center gear. If your trip involves third-party engineers, plan things to do in case their flight is delayed or they get moved to a different client for a day. Have flight, hotel, and other contact info handy in case you do need to extend your trip for a few days.
Keep good documentation
Documentation is important for an organized data center but it's even more important if no one will go back to your server room for months. Be sure to keep good notes of any network or power connections you made, what tools were at the data center and what you left behind, and depending on the sophistication of your monitoring you might even notice a few amber lights on servers or hard drives that you need to look into. If your data center allows it, you might even want to take pictures of the changes you made and of the overall look of the server room. It's certainly easier to describe your work when you have pictures to back it up, and it will help any new administrators who have to visit the data center later. Finally, keep track of any setbacks you run into so you can improve your plans for the next visit.
ABOUT THE AUTHOR: Kyle Rankin is a systems administrator in the San Francisco Bay Area and the author of a number of books including Knoppix Hacks and Ubuntu Hacks for O'Reilly Media.
|
|
|
|
|
|
