The Contingency Planner

By Don Edwards

The Contingency Planner, a title that varies throughout the business world (Disaster Recovery Coordinator, Contingency Coordinator, etc.), is one of the newest positions in both the Corporate and IS organizational structures. Emergency Planners have been around for a long time; however, not many graced the halls of large corporations until the early 1980’s. Even today, some organizations refuse to accept the position of the Contingency Planner as a permanent one. However, much is changing in Corporate America - especially following the natural disasters of ’89 and ’90 that left a few companies attempting to recover without adequate backup plans.
Even though “we’ve come a long way baby,” there is still a long way to go before Contingency Planners hit the top of the org charts. Somewhere between three to seven years ago, most of us began our careers as bona-fide Contingency Planners. Some of us had planning backgrounds, some technical writing, others had DP backgrounds. But few of us really had a genuine understanding of disaster recovery.
For many a contingency planner, the career began with no script, no procedures, no direction, no budget, not much pay, and most importantly, no staff! Many of us had to begin by selling the idea of a hot-site contract to senior level management; hundreds of thousands in cost per year with no visible payback! And after months of work justifying the hot-site, our next step was to develop a test plan and test at a hot- site or backup site.
The enthusiasm the first year of testing was phenomenal. People begged us to get involved. The first year’s testing for most of us was great and we thought this whole contingency thing was a piece-a-cake. As the second year came along, management thought “why bother?” We had the operating system functioning, some telecommunications, a few applications had run successfully - what more could we need? That’s when we--the contingency planners--learned that too much early success was indeed a bad thing.
The greatest obstacle in the way of success of any contingency plan is not having complete commitment. If you don’t have that commitment, you don’t have a plan. To get that commitment, management must understand the scope of what needs to be done to develop a completely functional contingency plan. After awhile, management realized that the early successes in testing were due primarily to goals that were reachable and that the ultimate goal of 100% recoverability was still far from becoming reality. I remember early on making predictions that our entire plan would be fully tested and documented requiring only periodic maintenance within two years. Not quite! And, as most of us now know, the further we dig into the puzzle of recovery the dirtier it gets! We, as Contingency Planners, found that we were not backing up all the datasets necessary to recover from a complete destruction scenario. We also found that some of the so-called standards that existed for years were not necessarily being used by the development staff. Most of our application rerun documentation relied upon datasets that existed on DASD, but they were not backed up to tape. We also found that human error becomes an all too frequent enemy that can cause an entire test to fail; not to mention what might happen in an actual emergency. We learned the extreme differences between tests involving one team working single shifts over several days and several teams working continuously for 48 or more hours. And most importantly, we learned that the worst four-letter word in a Contingency Planner’s vocabulary was “assume.”
As Contingency Planners, we have learned through the years that we must write ourselves out of our plans because we are not the doers. Our company’s personnel must perform the recovery operations and our job ends when the emergency begins. That leads us to our greatest challenge - getting people involved! With no staff we must find ways to challenge the day-to-day employee to think contingency when developing new applications, installing new equipment, and planning for the company’s future. They hate to see us coming because they know it means more work piled onto their already busy schedules. Contingency Planners have to be the pushiest, friendliest, most helpful and persevering employees of any organization. Without those characteristics, we risk apathy and dissention among the only people who can make the plan work - the employees.
We must also be salespersons, or how else could we convince management that the latest piece of the recovery puzzle is essential to the success of the entire plan and without it the plan is doomed to failure? We must also be strong enough to be criticized for test failures, pushing too hard, and for not being the technical expert. A Contingency Planner comes in contact with each and every manager up the entire ladder clear to the Board of Directors. And, in most organizations, he or she is probably one of the lowest ranking employees with that level of responsibility. Most of us live without the backing needed to perform this job adequately, and yet we still manage somehow to make progress in the development of our company’s contingency plans.
As Contingency Planners, we must coordinate a plan to rebuild in a few hours what has taken many years to construct. We must have a better than average knowledge of every business function, the applications that serve those functions, and the processes used to rebuild them following an outage. We must be able to communicate on all levels both within and outside of the organization.
Sound impossible? Well, it very nearly is. I take my hat off to all Contingency Planners of the World - because our job is certainly one of the great challenges left on this planet! Good luck. Oh, and by the way - I wouldn’t change this job for another if my life depended on it!

Don Edwards, CDRP, works at the Puget Sound Bank in Tacoma, WA.

This article adapted from Vol. 4 No. 2, p. 59.

Disaster Recovery Worldİ 1999, and Disaster Recovery Journalİ 1999, are copyrighted by Systems Support, Inc. All rights reserved. Reproduction in whole or part is prohibited without the express written permission form Systems Support, Inc.