DRJ's Spring 2019

Conference & Exhibit

Attend The #1 BC/DR Event!

Spring Journal

Volume 32, Issue 1

Full Contents Now Available!

The physical cleanup of the buildings and contents affected by the Chicago flood of Monday, April 13, 1992 was “restoration” in its most simplistic, yet detailed form. Simplistic in the sense that it was very labor intensive as compared to a high technology disaster, yet detailed in its attention to project management and proper health and safety procedures.

Have you ever wondered what it takes to restore a major metropolitan area to normal after a disaster like Hurricane Andrew or the Chicago flood? In any major city or rural township the list of restoration tasks is endless.

These disasters caused water damage from heavy rainfall or flooding. But water damage can also occur from fires, broken sprinkler systems, and frozen pipes that burst, and the specialists that meet this challenge must work quickly and skillfully to prevent further water related damage. The dehumidification business, under normal circumstances, requires dedicated, knowledgeable personnel trained to use and maintain a variety of drying equipment to meet the 24 hour-a-day, 365 days-a-year demand and, of course, in the event of a hurricane, tornado, or earthquake, these requirements are doubled.

Dehumidification of moist air is a somewhat new technology in the field of disaster recovery. In the past, most large-scale drying jobs were handled with open windows and blowers to dry out saturated areas. Tightly sealed buildings of today require a high tech approach to moisture removal. State of the art powerful commercial drying equipment can now convert a vapor (i. e. humidity) into a solid and remove as much as 200 gallons of water a day, saving a building and its contents with great success. Knowledgeable dehumidification specialists handle everything from modern high-rise commercial structures housing high-tech communications systems, computers and electronic storage media to historic mansions containing beautiful, old wooden floors and antiques, not to mention irreplaceable paintings, valuable documents and archives.

Let’s examine two extreme comparisons: an historic structure and a modern high-rise building. One might pose an argument that the methods for drying both are essentially the same. Not so. Although the equipment and instruments perform basically the same functions, the drying procedure is critical to the characteristics of the structure itself, and these characteristics warrant some discussion.

A good historical example is an antebellum church that suffered devastating water damage during Hurricane Hugo. Its architectural features—attics, crawl spaces, the areas beneath the altar and around the massive pipe organs became collecting pools for moisture and presented some particularly difficult dehumidification problems. Drying a modern building with synthetic carpeting, vinyl wall coverings, and sheetrock is one thing; it’s quite another to dry the framework around a stained glass window, ornate wood trimmings, and plaster that is more than a century old.

Each type of wood in the structure had its own unique density, grain, and porosity which affects the moisture content of the fibers. Aging and exposure to the elements also added to the difficulty of drying the wood in this beautiful old church. Therefore, special attention was given to restoring the wooden fibers to their own correct level of relative humidity, and this was done in precisely staged increments of time, which allowed the wood to shape itself “comfortably” and as naturally as possible in relation to its environment.1

Compounding the inherent material problems are aesthetic considerations and priceless historical value of the structure and its contents. Knowing what to do, and how, will determine the difference between restoration success and costly replacement. Therefore, it is imperative that dehumidification specialists be sensitive to the importance of preserving the original state of each item with which they are entrusted.

A modern structure, on the other hand, poses its own set of drying problems: steel instead of wood structural elements; floor and wall coverings of synthetic fibers and materials that may create toxic fumes when damp; and high-tech communications and electronic equipment which are often hidden in subfloors, walls and ceilings. Temperature and humidity as they relate to healthy indoor air quality and sick building syndrome is undoubtedly another major consideration.

The only constant in the dehumidification process is that moisture seeks, and will transfer to, a dry spot. This means, literally, that if there is any moisture at all in the environment, whatever is dry will attract moisture, (i.e. upholstery, carpets, walls, paper, etc.). The right dehumidification equipment, response speed, and constant monitoring, will insure the restoration success, and remove the threat of exorbitant replacement costs and an unhappy customer.

It is not surprising then that the demand for dehumidification services is rising. Here are some other important reasons why:
1) The cost of replacing building materials, furnishings, carpeting, wall coverings, and labor has increased dramatically, forcing builders and insurance carriers—particularly those who deal in large commercial structures—to consider other, less costly alternatives.
2) Dehumidification takes less time than replacement and frequently requires little, if any, disruption of the physical environment—a feature that is particularly attractive to businesses since lost wages are not covered by most insurance policies. Normal business activity can continue without interruption or can be resumed in a matter of a hours, not weeks.
3) Dehumidification reduces the risk of many of the health related problems associated with indoor air quality. Recent studies show that high humidity provides a favorable medium for the survival and rapid growth of such biological contaminants as bacteria, viruses, fungi, and mites, and that excessive moisture can create certain chemical interactions with substances that are used in building and decorating materials. These scenarios are potential time bombs that insurance companies, property managers/owners and risk managers must eliminate.

Authorities are now exploring if, and how, the severity of natural disasters will intensify in the wake of such catastrophic events as the eruption of the Mt. Pinatubo volcano. Recent studies suggest the effect on the United States of the volcanic ash cloud formed by Pinatubo’s volcanic debris can precipitate a cooler winter, resulting in frozen pipes, thereby producing water damage situations.
Weather related disasters, the daily mix of broken sprinklers, fires, and floods, the increasing focus on sick building syndrome2 and the importance of clean, healthy air present many possibilities for the application of dehumidification services. Each and every person must take an active part in learning what constitutes a healthy environment for ourselves and those who follow. Awareness to the benefits of dehumidification services clearly defines the difference between a planned program of preparedness and an unhealthy disaster.
These observations lend significant weight to the claim that there is more to the business of drying than meets the eye. In this business there is no template; the conditions dictate the approach, and the conditions are like the New England weather: wait five minutes and things may change.

Thomas Zoll is founder and President World Wide Drying, Inc., a Massachusetts-based corporation providing dehumidification and restoration services. He has over 25 years of experience in dehumidification, disaster recovery and business continuity planning. His current specialization includes the complete dry-down of electronics, telecommunications systems, computer systems, books and documents.

This article adapted from Vol. 5 #4.

On the morning of July 30, 1993, at approximately 7:00 a.m., employees of the Chesterfield Bank were getting ready to go into work as usual. Little did they know this wasn’t going to be a typical Friday at the bank. Bank officials had already been contacted by the Chesterfield authorities that a potential for a disaster existed.

Comdisco Disaster Recovery Services (CDRS) has always been proud of its industry leadership position. In that role, we continually strive to raise corporate awareness of the potential for catastrophic events to occur.

We have learned that these situations, usually unforeseen, occur in the most unlikely of circumstances often crippling an organization’s ability to function effectively. We ourselves are not immune to these unpredictable events, as we found out on October 31, 1991.

CDRS has led the industry in supporting customers who have had disasters which required invocation of recovery plans and operations.

We have learned a great deal from supporting the recoveries of more than 66 actual disasters and more than 13,000 subscriber tests. One of the benefits we provide our customers is the sharing of experiences gained in supporting these recoveries.

In the mid-1980s, CDRS began sponsoring seminars and subscriber briefings to share the experiences learned in supporting the most publicized disaster to date: the Montreal fire at Steinberg Corporation’s headquarters. When the industry experienced its first multiple disaster, as a result of the Chicago floods in 1987, CDRS continued sharing information through a similar set of seminars and briefings.

In 1989 and 1990, the San Francisco earthquake and New York power outage produced unprecedented concurrent declarations, eight and 12 respectively, for CDRS.

Once again, CDRS provided speaker platforms at our user conference and supported industry conferences on the subject of experience. We have, and will continue to learn, along with our customers, during these situations.
Each event, or set of events, continues to mature the industry. The Chicago floods demonstrated that a commercial vendor could support multiple, concurrent declarations.

The Hinsdale fire at the AT&T switching station provided insight into the importance the communications industry plays and the impact of an external disaster. San Francisco and New York dealt with user area recoveries in conjunction with or separate from data processing outages.

CDRS recoveries in Paris, London and Singapore highlighted the global nature of our industry. A 203 day customer occupancy of our Cypress, California facility demonstrated the viability of outfitting and using a cold site for an extended period.

Throughout these events, CDRS has continued to focus our subscribers on the fact that disasters do in fact occur and that everyone is vulnerable.

During the last several months, the industry has learned that “everyone” includes vendors. This situation also confirms that a vendor strategy of multiple centers and transportability for networking is a critical selection criteria.
October 31, 1991, (Halloween) was an especially frightening night for CDRS’ Carlstadt, New Jersey complex.

The Carlstadt complex is made up of two separate buildings, interconnected for channel connectivity. Building A, located at 430 Gotham Parkway, houses our Continuous Availability Services (CAS) products. Building B, located at 480 Gotham Parkway, contains our traditional computer Recovery Centers (hot sites) for IBM and Tandem users. Let’s take a look at the events that occurred.

At 5:00 a.m. Thursday, October 31, 1991, extremely high tides, as a result of a tropical storm off the Eastern Seaboard, caused flooding in the Carlstadt area, including the parking lot adjacent to CDRS’ Carlstadt, New Jersey complex. These tides were the highest level recorded in the past 30 years.

The flooding was a direct result of a failure in a water control system. Essentially, an earthen dike used to contain and divert tidal waters was damaged prior to the storm and allowed flooding to occur. Subsequent repairs and comparable tide levels confirm that repair action has eliminated the problem.

As a result of the flooding, dual commercial power fed into the CDRS Carlstadt Complex were interrupted. At no time did water enter or threaten to enter either CDRS building.

However, on the outside of the 430 Gotham Parkway building, externally mounted electrical switch gear was damaged by water.

Within five hours, commercial power was restored to CDRS’ 480 Gotham building, which houses our dual IBM 3090-600 backup offering, obviating the need to utilize diesel generators as originally planned. The 430 building was restored with diesel power by 11:00 p.m. Thursday and converted back to commercial power on Sunday, November 3, 1991.

Immediately upon identification of the problem, CDRS implemented its own recovery plan. Such plans exist for a potential interruption to any CDRS facility as an acknowledgment that no facility, customer or any vendor, is exempt from a problem.

As part of the plan, CDRS’ CDRS NET architecture initiated instantaneous, automatic rerouting of the backbone network around the affected facility, ensuring integrity of the any-to-any facility linkage.

We also rerouted the CDRS disaster declaration hot line number and other Carlstadt phone lines to our North Bergen, New Jersey facility for uninterrupted phone coverage.

Also, CDRS’ immediate steps included the declaration of a disaster for our CCSC business unit, which provides the CAS services, into one of the four recovery centers in the 480 building. This action, as with the 66 previous customer disaster declarations, necessitated rescheduling five customer tests. This was in full accordance with long-standing CDRS policy.

Separately, our remaining recovery facilities were continuing to support testing and stood ready for customer disaster needs. CCSC officially removed itself from disaster status on Sunday, November 10, 1991, returning the utilized hot site to full testing and recovery availability.

Following an in-depth review, CDRS has committed to undertake significant steps to ensure against a potential reoccurrence. We believe the steps we will undertake to be at a level of redundancy and protection commensurate with the criticality of our services.

First, a documented plan to construct a retaining wall with sump pumps completely isolating the property has been initiated. This will be a fail safe backup to the earthen dike.

Second, we will institute our own program to ensure that proper monitoring and maintenance procedures for the tidal control system are documented and followed by the appropriate owners. Discussions are underway and 100 percent cooperation has been assured. This will allow us to ensure we are in control of the situation at all times.
Third, a secured, waterproof enclosure to the power system is being constructed as an additional fail safe measure. This enclosure, including dual sump pumps on battery backed up power, will ensure protection to the switch gear. In addition, a water detection system will be added as an early warning and protection capability.

We also believe we, and the industry, learned a valuable lesson: No one is immune to an unplanned outage.

Finally, a solution for adding diesel power generation will be implemented. Engineering studies and preliminary local zoning approvals have taken place, paving the way for an expeditious completion.

These steps, we believe, will ensure that a reoccurrence is a virtual impossibility. We have already begun construction of these fail safe steps. Due to varying construction schedules, completion will take place over the next few months.

It is important, we believe, to step back and consider the experiences we have gained from supporting ourselves and our customers during these disaster recovery activities.

First, we have learned that clear, concise communications to an organization’s customers and the press is of paramount importance. As is often the case, several trade journals quickly picked up on our disaster recovery story. In spite of our attempts to accurately portray the situation to the journalists, a great deal of contradictions and inaccuracies were reported. This even resulted in one of our subscribers being grossly misquoted in terms of their opinions and plans following the event. This subscriber has been extremely helpful in assisting us with other customers who were concerned about the out-of-context comments printed in the trade press.

Second, we were gratified to see the value of our CDRS NET Any-to-Any architecture. We believe this strategy enhances our ability to sustain an outage in the system with minimal subscriber impact. It should be noted that the Carlstadt recovery center utilized to support the disaster only represents 15 percent of our North American supply of capability, leaving more than adequate protection for our remaining subscribers. Our philosophy has been, and always will be that numerous, geographically dispersed facilities ensure one outage could never cripple our business and/or unduly expose our customer base.

Third, we have reconfirmed the value of having a documented, tested plan for recovery. The ability to relocate and restart our CAS products was directly related to their preparatory steps.

Fourth, all organizations should investigate the option of having cellular phones available to facilitate communications. With no power, and a commensurate loss of our PBX, cellular phones provided a valuable communications lifeline.

Lastly, CDRS, in conjunction with its consulting team, has developed and implemented a set of procedures to evaluate all of the physical components of each recovery facility.

These procedures, included in our Prevent! product, are thought to be the most exhaustive proforma in the industry, incorporating over a decade of personal experience and in-depth collaboration with clients and vendors concerning areas of uniformity and compliance.

The objective is to provide an extra level of protection for each facility and produce a risk analysis and a set of physical specifications down to CAD based floor plans and schematics for all hardware and power components. This will enable the development of a preventative program that minimizes the likelihood of a disruption and reduces the impact of a disaster.

In summary, CDRS believes the interruption to our Carlstadt Complex was an unfortunate circumstance. We believe we responded with swift and effective actions and minimized CDRS customer exposure. We also believe we, and the industry, learned a valuable lesson: No one is immune to an unplanned outage.

John A. Jackson is executive vice president of Comdisco Disaster Recovery Services, Inc.

This article adapted from Vol. 5 #1.

The underground flooding that paralyzed the Chicago downtown business district in mid-April was little more than a drop in the bucket for organizations with tested business resumption plans in place.

While many companies were unprepared for such a disaster, quite a few organizations, with tested recovery plans in place and crisis management teams ready to act, managed to keep their heads above water during the industry’s biggest crisis to date.

The flooding, which caused local utilities to cut off power to a 12-square-block area of businesses in Chicago’s “Loop,” and forced the evacuation of dozens of major buildings, prompted seven SunGard subscribers to declare disasters.

Five of these subscribers successfully recovered their applications at SunGard’s Chicago, Philadelphia, and San Diego MegaCenters, and two of these customers employed SunGard’s end-user recovery facilities. Two of the seven subscribers later decided that the problems they incurred did not warrant their use of an alternate processing site.

An additional 10 customers placed SunGard on alert during the course of the disaster situation, and their status was monitored throughout the crisis until downtown Chicago’s power instability problems were solved.

While the pace at SunGard was hectic, the situation remained firmly under control throughout the recovery, which began 7:30 a.m. on April 13, and lasted until midnight, April 27, when the last subscriber officially left the recovery facility. Effective crisis management, based on planning and teamwork, was the key.