When You Least Expect It ...
- Published on November 19, 2007
Then, suddenly, two major events transpired within a 10-day span:
- On Thanksgiving weekend, a major water main erupted in the parking area in our building, requiring the aid and assistance from city workers. In brief, water had penetrated several important closed rooms, damaging very important work documents, furniture, and stationary. The work documents were stored downstairs for safekeeping since they were regularly referenced and needed not to be stored offsite. The entire clean-up process took approximately four days to fully contain the damages.
- Ten days after this first incident, a fire alarm was set off in the server room at 1 a.m. Receiving the call from the building’s security, I immediately proceeded to call all necessary resources and made my way to the office. Without delving into great details, the fire alarm was set off because something had occurred with the main uninterrupted power supply (UPS) unit that feeds our servers. After the fire department forcibly entered the double-paned glass window by removing it from its hinges, the necessary resources were contacted by the building’s main security and all was contained within a matter of six hours.
I learned many lessons that bear mentioning. I will also describe some of the key events that transpired during these incidents, which may aid some practitioners what to do – and what not to do – in case of disaster.
Always Be Prepared … and Expect the Unexpected
I this lesson in a previous article, but I cannot bear stressing the importance of “expecting the unexpected.” For instance, having the contact information of your most critical resources at your fingertips saves much time and effort when you need it the most. You don’t want to be squandering for hours, trying to retrieve critical information that could have been easily referenced if you had taken the appropriate measures to do so in the first place. Basically, effective planning and preparation is key to an efficient recovery. Such pre-determined lists need to be part of an on-going maintenance process that should be automatic.
In the same light as above, ensuring that more than a single mode of communication is documented within vendor, client, and internal resource lists is vital. Having multiple ways to contact someone ensures a critical person may be contacted. Possessing only home phone numbers is not realistic in this day and age. Cellular phone numbers, pagers, personal faxes, Blackberry PIN numbers, and e-mails need to be incorporated in the documentation process. I learned this step the hard way.
Outdated vs. Updated Information
Relying on the work of others, or depending upon work that has not been updated for some period of time, cannot be part of an effective business continuity/disaster recovery plan. Assuming that something has not changed because it has never changed in the past is denying the inevitable: outdated information will result in frustration and lost time looking for an updated version. Periodically reviewing and validating all of your critical information, from seldomly used vendor lists to often changing internal lists, must be instilled in the maintenance process of the planning cycle.
As mentioned before, a first step in being well prepared is to properly document all information the corporation deems vital in case of a disaster. In the same vein, nothing can be further from the truth than possessing updated contact lists. Being thorough not only pertains to having all of the possible contact information from one key individual within a single function, it also means having multiple contact points for that same function. For example, I needed proper authorization to move all affected files during the flooding situation to a proper storage area with our third-party service provider in order to have them treated. I called everyone down the hierarchy within the respective department – approximately 15 people. After going through the whole list, I finally spoke to the administrative assistant. I learned you have to have more than one contact person per department.
The Post-Mortem Report
One of the most important processes stemming from the aftermaths of a disaster is the documentation of what transpired. Consolidating and validating the chronology of events that occurred is a tremendous task. After doing so for both events, the building’s reports did not coincide with what we had. Coordination was needed from both sides in order to collaborate and corroborate the sequence of events. Such a process needs to be properly determined and documented beforehand. Do I start with the cause of the events? Should I then process with the accounts from each party? I learned you cannot simply rely on one version of the story to recapitulate and comment on such disasterous events. A consensus needs to be discussed amongst all parties involved to establish clarity and reliability in the results.
The Road Ahead
This article adds credibility to the assertion that a disaster can – and does – occur at any moment in time. Having nothing chaotic happen during the first 18 months as BC coordinator and then, suddenly, being affected by two incidents within two weeks proves that point. Nevertheless, a positive outcome from these incidents is the reassurance and likewise acceptance of having a BC plan and full-time coordinator in place.
I was encouraged and honored for a job well done. Upper management felt justified in making the decision to implement a business continuity program. The events that transpired, and the handling of those events, have likely ensured the continued existence of a BC program for many years to come.
Michael Barbara, CBCP, is the recently-appointed business continuity coordinator at a law firm. He has been employed there since March 2003. He has previously worked as a business analyst and is currently pursuing a master’s degree in administration.