Disaster Recovery Journal - Spring 2003 Issue-Why Some Recovery Plans Won’t Work

Return to the Spring 2003 Index

DISASTER RECOVERY
JOURNAL

P. O. Box 510110
St. Louis, MO 63151
(314) 894-0276
Fax: (314) 894-7474
Internet www.drj.com
E-mail drj@drj.com

PUBLISHER &
EDITOR-IN-CHIEF
Richard L. Arnold, CBCP
richard@drj.com

SENIOR EDITOR
Janette Ballman
janette@drj.com

MANAGING EDITOR
Jon Seals
jon@drj.com

COPY EDITORS
Richard Sandhofer
richards@drj.com
Pamela Clifton
pamelaclifton@hotmail.com

ADVERTISING
Robert Arnold
bob@drj.com

_____________

Corporate

President/CEO
Richard L. Arnold, CBCP
richard@drj.com

Vice President
Robert Arnold
bob@drj.com

CONFERENCE COORDINATOR
Patti Fitzgerald, CBCP
patti@drj.com

CONFERENCE REGISTRAR
Merce Knese
mercedes@drj.com

CIRCULATION
Laura Baugh
laurab@drj.com

EXECUTIVE COUNCIL
Patrick Corcoran, IBM Bus. Cont. & Rec. Services
Jeff Dato, MBCP, KPMG
Edward S. Devlin, E.S. Devlin & Associates
Judith Eckles, SunGard Availability Services
James Hammill, CBCP, JMH Consulting Inc.
John Jackson, Independant

INTERNATIONAL
CONTACTS
England: Thom Hetherington
Business Continuity
Phone: 0161-237-1007
thomh@tempus.demon.co.uk

Australia: Anthony J. Harvey
Journal of Business Continuity
Phone: 0011-613-953-0055-8
fax: 0011-613-953-0528
sector@notability.com.au

Japan: Shinji Hosotsubo
Quake Japan Co., Ltd.
Phone: 03-3215-2880
fax: 03-3215-2881

Brazil: Jose Carlos Ferreira
Disaster Recovery Mercosul
Phone: 55 11 3666-9506
conc2000@uol.com.br
www.drms.com.br

Click Here for a Printable Version

Common Flaws

Why Some Recovery Plans Won’t Work

By JUDY BELL, CEM

The Whittier earthquake struck as I was coming down an off-ramp on my way to work in Pasadena, Calif. I first thought I had four flat tires. Then, as I looked around me, I began to realize that one of the disaster scenarios we had imagined was beginning to occur.
As an operations division manager for Pacific Bell and chair of the Los Angeles Emergency Operations Committee, one of my first actions was to retrieve a copy of our business continuity plan, a three-inch ring binder, which I kept in the trunk of my car. The committee I chaired was composed of 22 top managers within the region, each representing a different discipline. We had formed a task force just one year earlier to develop plans specifically to react to an earthquake, and we had put many elements of our plans in place, most of which were documented in that binder.
Fortunately for me, this earthquake was moderate as earthquakes go – only $1 billion in damages and seven fatalities in southern California. But before the morning was out, I had abandoned that binder because it was so distant from the realities we faced.
As a direct result of our experience that day, Pacific Bell re-examined all aspects of its disaster preparedness plans. We redefined the role of the emergency operating centers, and we began a program to train employees and to regularly test our capabilities. All of this preparation paid off two years later when the Loma Prieta Earthquake struck the San Francisco area. By that time, the emergency operations committees were fully prepared because their drills had so closely depicted what to expect. Using their revised business continuity plans, they were able to assess the damage quickly, dispatch resources where they were needed, and put controls into the network to minimize congestion. For Pacific Bell, its planning had paid off.
Having reviewed hundreds of business continuity plans since those days, I am frequently surprised to discover that many of the mistakes we created then are still being written into business continuity plans today. Considering the variability that is possible in disaster scenarios, it is surprising to discover that most plans that fail have similar flaws. Here are some common ones to look for:
1. Information not organized effectively. All of the planning in the world will not help if the information is not properly recorded. When the real situation arises, the wrong people may have the document, and those who have it may waste valuable time searching to find what they need. The real trick is assembling the right information for the right people in the right way.
I recently reviewed a company’s plan that had been prepared by a team of employees. My test for its practicality was to flip to the table of contents to see how quickly I could find the vital information needed to deal with life-safety issues. In this case, I couldn’t even identify who the incident commander was. What I found instead was a binder filled with excerpts of information from a variety of sources, with no common content or theme.
When preparing plans, it is important to organize the information in a logical flow of how it will be used at the time of an event. Information that is used only for planning purposes should be contained in a separate binder.
The size of your company will determine how many different documents you will need. If you are a large company with more than 500 employees and multiple locations, chances are you will need a variety of documents, including a key executives document and a company plan. The key executives document should be a small, brief, concise guide which tells top managers what to do immediately following a disaster. Any organizational scheme other than a sequential approach will not be helpful when the real situation arises, and documents for executives should always spell out who is responsible for what. The key executives document should also include a removable key information page that contains the current telephone numbers and alternate contact information for all key contacts that may need to be engaged immediately following a disaster.
The company plan is used primarily by department planners as a source for company policies. Key executives also use this to address long-term recovery efforts. However, unlike the key executives document, this document should create a clear picture to others of what the company policies are, and how the business will respond generically to a disaster. If your business is very small and located in only one building you may be able to create a single document that contains everything you need.
2. Format or software is too complex. Modern tools for generating business continuity plans are powerful and marvelous. They enable small planning teams to create electronic documents very quickly and consistently, reminding them of areas they may have overlooked. However a tool for generating plans will become a significant barrier for you when disaster strikes if knowledge of the generation tool is required to access the information stored within your planning documents. Will the computers that house the plans be accessible to those who need them? Will the applications needed to view the plans be up and running? Will the current versions of the documents be available? Will your responders know how to find what they need in documents that have been created by someone else? Have key people in your organization rehearsed the plans that have been created? If you cannot answer positively to each of these questions, you have symptoms of one of the most common flaws in today’s business continuity plans – document complexity.
Regardless of how it was created, you should always have current copies of the plan stored off-site, in paper format, where they can be easily accessible.
3. Plans are too generic or too detailed. Plans that are too generic almost always look good on paper, well balanced and formatted. Perhaps they are the product of a project team’s “cookie cutter” approach or delegation of the job to planners who did not really appreciate the importance of their tasks. Inevitably when disaster strikes, generic plans fail because they are based on assumptions that prove false or incomplete at execution time. While generic plans can be excellent tools for use during the planning process, they must be carefully scrutinized by all of the stakeholders and they must be tested regularly to make sure they work.
Conversely, plans that are too detailed can be equally troublesome when disaster strikes. Two kinds of errors can cause this flaw in your plan: either the business impact analysis did not identify the true “worst case” scenario for your company, or the people preparing the plans documented critical functions too finitely. Inevitably, the disaster that actually occurs will not be a good match for the disaster that was anticipated, and the plan will have serious gaps. Therefore, it is always best to focus initially on the scenario that represents the worst case for your company.
4. Alternates not identified. Like many others, when the Whittier Earthquake struck, I immediately attempted to use the cell phone in my car to contact the managers at our key locations. Since our building was evacuated, I did not have access to our normal communications. Of course, I could not get through on my cell phone because the networks were jammed, and I did not have a list of other phone numbers to reach them, or the people they had designated as their alternates. Only later did I learn that while everyone else was struggling to get through, the amateur radio operators within our company had quickly relayed damage information to each other. Thereafter I always carried current information, not only for my key contacts, but also for whomever they had designated as their alternates.
5. Information is not up to date. In a large organization, a disaster recovery plan on the shelf becomes more out of date almost every day, primarily because of changes to personnel and vendor contact information. A good disaster recovery plan should include a continuous process for validating the contact information for the people who will be needed when disaster strikes, and it should provide alternate means for reaching them should the primary contact approach fail.
I share with you these common flaws in business continuity plans I have observed over the years. There is no feeling scarier than facing the responsibility of managing a recovery effort, knowing your plan is inadequate for the task. By screening your plan to detect these common flaws, you can greatly minimize your organization’s risk. In creating effective recovery strategies, planning is key!

Judy Bell is CEO of Disaster Survival Planning Network (DSPN) www.dspnetwork.com. She founded the company in 1988 after a 22-year career with Pacific Bell, where she was a division manager of network operations and chair of the emergency operations committee (EOC).

To comment on this article, go to 1602-03 at www.drj.com/feedback.