When the disaster struck, I went to Chicago to report on the damage and take photos. The other articles in this special report come from members of the disaster recovery industry who experienced the flood's impacts first-hand.
Although this issue of Disaster Recovery Journal had already been printed, mailing was halted so that we could bring you this special report on this major event.
This disaster will have a far-reaching impact on disaster recovery. Many companies in Chicago were forced to implement their disaster recovery plans. 230 buildings lost power because water threatened their underground power sources. For many disaster recovery planners, this was the first real test of their work. Hopefully we can learn from their experiences.
Report from Michael Gomoll Chi/Cor Information Management, Inc.
Construction workers had been working along the Chicago River for some time.
One of the projects included placing support pillars into the Chicago River
bottom. It is theorized that during the placement of one of these pillars,
a portion of the turn-of-the-century coal delivery system was damaged. A hole
the size of a large automobile formed in the bottom of the river and punctured
the tunnel ceiling. Exactly when the rupture took place is unclear, but on
the morning of Monday April 13th several Loop office buildings began to report
significant amounts of water in basement and sub-basement facilities. The
flooding was caused by massive amounts of river water pouring into the maze
of underground tunnels. The tunnels led directly into the basements of many
of the older Loop buildings.
Marshall Field’s flagship store, located on State Street in the heart of
the loop, reported flooding in sub-basements two and three with water levels
reaching 40 feet. With heating and electrical systems located in these basement
areas, not to mention a substantial amount of valuable inventory, the threat
was significant.
Most of the City and County governmental buildings are also located in the
Loop. As with the Marshall Field's building, the City/County utilities were
threatened by extensive flooding. Valuable assets were also in jeopardy, but
in this case the assets took the form of valuable government records. The
records existed in hard copy and microfiche form and contained a wealth of
historical information about the nations third largest city.
Chicago’s financial district, including the Chicago Board of Trade and Chicago
Mercantile Exchange, was also threatened by the torrent of water. The threat
of the flooding came not so much from the water itself, but from the impact
the flood could have on the extensive electrical and computer networks. It
doesn’t require an advance degree in electrical engineering to imagine the
potential safety risks to office buildings, some over 100 stories high with
thousands of tenants, if the electrical system is comp romised. By mid-day,
the entire loop business district was evacuated.
CHI/COR Information Management’s corporate headquarters is located in the
heart of the loop and was also affected by the flooding. Located in the southwest
corner of the loop, adjacent to the Sears Tower, its offices were on the outer
edge of the affected area.
The first order of business for the Executive Committee was to determine
what business processes were threatened, and based upon a business impact
analysis, initiate the appropriate recovery steps. The Executive committee
highlighted the following recovery processes:
Here is how each of these areas was addressed.
Evaluate The Threat To Personnel
It was determined that the threat to personal safety was minimal at the
time of disaster declaration, though it was decided that eventually complete
evacuation would be a necessity.
The buildings elevator system was scheduled to be shut down by 1:30 p.m.
To help avoid a bottleneck at 1:25, all “non-essential” personnel were evacuated
in stages as their recovery functions were completed. All employees were kept
informed of evacuation alternatives and timeframes. By following the predefined
evacuation procedures and routes, all “non-essential” personnel were evacuated
well before the 1:30 deadline. This orderly evacuation helped make the subsequent
evacuation of the emergency staff quick and efficient when the time came.
Activate Customer Support Network Procedures
After all personal safety issues were addressed, the Customer Support Team
went into action. Their first order of business was to re-route all customer
support lines to a cellular telephone network. Once these cellular channels
were in place, the Customer Support Team worked in tandem with the Off-site
Facilities Team to ensure that support personnel would have continuous access
to their various support tools and databases.
Prepare Off-Site Facilities
Our plan called for the availability of personal computers at several off-site
locations for members of the support and development teams. Many support and
development personnel have access to PCs at home. The Customer Support plan
called for the staggered dispersion of personnel to off-site locations. This
tiered approach to the evacuation helped to maintain continuous availability
of all support functions, as no time was lost because personnel was in transit.
The transportation of support personnel was co ordinated by the Facilities
Evacuation Team.
Back-up and Secure Information Systems
The Data Center Team immediately instituted the necessary back-up and protection
of critical business applications and data. The back-up of the entire day's
activities was initiated and that information was available should the need
arise to re-locate to the hotsite facility. All systems and equipment were
then secured to protect against any threat caused by the flooding and possible
electrical problems.
CHI/COR President Rick Effgen commented “We were fortunate in many ways.
We had sufficient warning time, sufficient evacuation time, and the availability
of all of our recovery teams. These factors, combined with extensive planning,
allowed us to ‘practice what we preach.’ We were able to continue all major
business functions during the crisis and return to full operations the following
business day”.
The ramifications of this disaster, both physical and financial, will be
felt throughout Chicago for a long time to come.
Report from John Jackson Comdisco Disaster Recovery Services
CDRS received its first declaration before 9:00 a.m. Monday. By mid-day,
12 firms had declared 18 individual disasters. The disasters included subscribers
from the financial, brokerage, government and services/distribution industries.
Additionally, by the day following the initial disaster, 17 alerts were pending
for 13 additional customers.
Comdisco supported 8 customers at its Wood Dale, IL site, 7 in Carlstadt
and North Bergen, NJ, one AS400 customer in Bridgeport, NJ, one customer in
San Ramon, CA, and one in Alsip, IL.
Several subscribers utilized newly configured Workarea Recovery Centers
to provide telephone, personal computer and office space solutions for displaced
workers. For all vendors, this disaster called for more workarea recovery
space than previous disasters, reflecting more comprehensive business planning,
rather than just planning for data center recovery. “This is the largest recovery
in our history,” said Comdisco President Ray Hipp.
The disaster was essentially power related, as no actual computer centers
were destroyed by the flood. Lights and air conditioning in the work areas
are out, so personnel can not occupy the work space.
Once power is restored, many buildings will attempt to isolate themselves
from the problems and pump out the water. Efforts to dig a connection from
the freight tunnel to Chicago’s Deep Tunnel water retention system are projected
at 8 days or longer. The entire freight tunnel may be in some danger of collapsing,
and a further investigation will survey the total infrastructure of the downtown
area for other instability.
Unlike other disasters such as the San Francisco Earthquake and the New
York power outage, this was not a brief event followed by restoration. This
disaster could drag on for weeks. Just fixing the first part of the problem--stopping
the flow of water--could take a week.
Report from Barney F. Pelant Barney F. Pelant & Associates The story began last year when contractor Great Lakes Dredge & Dock Company
of Oak Brook, Illinois installed new pilings around the Kinzie Street Bridge
along a branch of the Chicago river that wraps around downtown Chicago. Speculation
is that the new pilings, which were driven down into the muddy depths of the
river at an unapproved location, ruptured a turn-of-the-century tunnel system.
Investigation of the causes will continue for months.
The 62 mile tunnel system was built to support rail and coal supplies to
the many buildings of the city. Today the tunnel is used to distribute various
cable systems, however the older buildings of Chicago have bulkheads accessing
the system. On January 14, 1992, two cable workers from Chicago Cable Television
came across the visible cave-in that was occurring in the tunnel at the Kinzie
Street Bridge. The concrete was breaking away and mud and silt were coming
through the openings. They recorded this on video and attempted to get through
the bureaucratic offices of the city to notify the Department of General Services.
The cost to repair the cracked opening was estimated by city officials at
around $10,000, but actual repair bids exceeded that amount. Consequently,
additional estimates were being requested, and ironically, a meeting to review
what should be done was reportedly scheduled to take place a day after the
tunnel ruptured.
Days after the disaster struck, the city remained crippled by the slow process
of draining millions of gallons of water from the tunnel system and two to
three stories of basements that support many of the high-rise buildings in
the city. It is here that we find the power distribution, cooling and boiler
plants essential to the functioning of the buildings. Tons of materials, including
broken concrete, mattresses, gravel and "rapid set" concrete have been dumped
into the river atop the opening to try to sto p the flow of water. But all
of this was of little help, as the water continued to rise in the buildings.
The city was declared a federal disaster area as federal, state and local
officials are working feverishly to solve the problem.
For the many businesses in the downtown area with major corporate computer
centers, it brought the biggest declaration of disasters yet to hotsite recovery
vendors.
This was the largest disaster ever for Sungard Recovery Services, which
had its first declaration of disaster at 7:30 a.m. on Monday. Seventeen Sungard
customers were affected.
The greatest financial impact was felt by the Chicago Board of Trade, which
shut down completely on Monday, and resumed trading only at small volume in
subsequent days. An estimated 25 billion in trading of the 36 products handled
by the CBOT were lost on Tuesday, before limited trading was re-instated on
Wednesday.
Although CBOT still had water cascading into its basements on Thursday,
it was conducting limited trading with darkened corridors and a limited number
of elevators and computers. This impact was felt world wide.
The biggest financial issue yet may be insurance, as this is not being considered
an “act of God” flood, eliminating coverage for many organizations. Estimates
are ranging in the $100's of millions in losses.
Business recovery centers like MEDS in Chicago were in immediate use as
emergency operations centers for key operations. On Thursday, McCormick Place,
one of Chicago's largest conference centers, offered its entire conference
space for one month as a free business recovery center for affected businesses.
The Chicago Transit Authority transit system of subways that support the
city have been closed down, with re-activation not expected for weeks. A parking
ban was placed on the entire downtown area to allow the access needed by the
many emergency vehicles being used to provide temporary power, and to pump
out water into the storm drains.
Several city blocks went without power for the first couple of days, however
by systematically cutting off the circuits to the flooded buildings, other
buildings have been brought up. Two of Chicago's largest department stores,
Carson Pirie Scott and Marshall Field's were both affected. The 3,000 hourly
employees at Field's and the 2,000 at Carson's were paid for work on Monday
if they were scheduled to work, but won't return to the payrolls until the
stores reopen. Employees at Filene's Basement have been laid off until the
store reopens.
The recovery in Chicago will be tentative for a long time to come. Simply
stopping the flow of water has been a major engineering challenge, and removing
water from the tunnels and basements has stopped until the primary problem
can be resolved. Complete blocking of the ruptured tunnels may take 10 days
from the initial disaster, and draining the tunnels will probably take an
additional two weeks.
The underground flood of Chicago on Monday, April 13, 1992 is proving to be one of the worst business disasters ever. It is exactly the kind of event that disaster recovery planners prepare for, but hope they never have to experience.
Reacting to the Flood
In 1899 the city of Chicago started work on a series of interconnecting tunnels
located approximately forty feet beneath street level. This series of tunnels
ran below the Chicago River and underneath the Chicago business district, simply
known as The Loop. The tunnels housed a series of railroad tracks that were
used to haul coal and to remove ashes from the many office buildings in the
downtown area. The underground system served Chicago well through the 1940’s
when other power sources replaced the coal furnaces. These tunnels went forgotten
until April 13th, 1992.
Evaluate The Threat To Personnel
Activate Customer Support Network Procedures
Prepare Off-Site Facilities
Back-up And Secure Information Systems
Chicago Businesses Forced to Declare a Disaster
At 5:57 a.m. on Monday, April 13, 1992 a building engineer at the Chicago
Merchandise Mart found water flooding the third sub-basement of the building.
Little did he, or numerous Chicago Fire Department and City workers on-site
by 6:10 a.m. know, that a hole the size of a car had ruptured in the restraining
wall of a freight tunnel located under the Chicago River. Fifty feet below
Chicago’s streets and below the Chicago River, a 100 year old, 60 mile long
freight tunnel interconnects the sub-basements of many Chicago office buildings.
This tunnel, no longer actively used for freight, serves as a conduit for
power, telecommunications and Cable TV cables. The rupture between the river
bottom and the tunnel caused over 250 million gallons of water to traverse
the tunnel into building basements, causing power to be shut off, buildings
to be evacuated, and businesses to c ease operations.
The Worst Disaster in Recent History
Trends may start on the east and west coasts (i.e. Hurricane Hugo and the
Loma Prieta Earthquake of 1989) in our country, but when it comes to doing
it bigger and better, don't count out the midwest. On Monday, April 13, 1992
began what has become the biggest business disaster to face us yet.