Schwab's Success Story

By Richard Newman

SAN FRANCISCO—Still basking in the success of his firm’s complete recovery from the 1989 San Francisco earthquake, Jeffery Lyons, Disaster Recovery Planner for Charles Schwab & Co., Inc., recently agreed to an interview with DISASTER RECOVERY JOURNAL. The following pages reveal some important aspects of the contingency plan they developed, some examples of how that plan held up during the actual emergency, and also some considerations for the plan’s continued development in the future.
“We were fortunate,” said Lyons. “We’d been running quarterly tests with our systems people and about a hundred users and had just finished a systems test at our outside vendor’s site about a week and a half before the [Tuesday] quake.” Indeed, the firm was so thoroughly prepared that, despite power outages, limited or no access to several buildings, and sporadic communications lines, Charles Schwab & Co., Inc. opened for market Wednesday morning “almost as if nothing had happened,” Lyons said.
Lyons and his colleagues certainly deserved the success they brought to the company. Five people had worked full-time for a little over a year during the planning phase, preparing for just such an emergency. The staff was also backed by a substantial budget and an extremely supportive senior management.
Lyons outlined some of the crucial aspects of the contingency plan. “The backup generators,” he said, “had full power in five to seven seconds, with no noticeable interruption.” After a building inspection and safety checks within the first hour, all non-essential systems were powered down so as not to overload the generators. It also proved necessary to rotate outages of certain equipment in order to maintain essential functions without causing an overload.
Since no one, including PG&E, knew when power would be restored, arrangements were then made to relocate employees to a nearby cold-site. The vendors at this site were capable of providing ample space, and the necessary supplies could be taken from a warehouse where they were stored nearby.
Employees were prepared for this relocation since ten days earlier they had successfully connected this site to the East Coast hot-site. At the vendor’s site they had also tested scripts for over 200 business functions.
Although linking the two backup sites alone was feasible, Schwab decided during the emergency to run the systems parallel and link the nearby cold-site to their headquarters production system. This extra precaution proved costly, but, as Lyons affirmed, “It was worth it.”
The Business Resumption Plan provided names and numbers of key personnel and detailed department lists of job functions and their criticality. From these extensive spreadsheets Schwab decided which non-essential operations to redirect and which employees to relocate to the time-critical operations which were to take place at the cold-site. The plan also contains information on how to notify branches of the emergency and provides a list of key personnel during an emergency.
Several employees were also needed to deliver the backup transaction tapes to the East Coast hot-site. Since all the airports in the area were out, a jeep was arranged to deliver them to Hayward, where they then chartered an aircraft for the first part of their journey. Lyons and his colleagues compiled telephone numbers of all surrounding airports, their charter information, runway restrictions, and the names and numbers of the airport managers. This important section of the plan also listed hotels and car rental information.
The Emergency Control Center contains an extensive employee locator file. These numbers are sorted by department as well as by area code, which proved useful since the sporadic phone lines often prevented calls within the area of the branch office, only permitting those outside the area. Frequently, personnel were forced to call outside the area in order to relay information to those within it.
The Emergency Control Center also provides extensive information on operational requirements and strategies, resources, telecommunications, and emergency financial arrangements. In addition, organizational charts provide information on management and staff backups and reports.
This seemingly excessive organization carried its privileges, for Schwab was frequently the only firm allowed access to certain closed buildings. As the security guards reportedly explained to another firm, “They have a plan. They know what they’re doing, they know exactly what they need, and they’re organized. If you want access, get a plan.”
Lyons praised the amount of assistance that the firm was able to provide for its employees. By Wednesday evening the Human Resources Department was working closely with employees who lost their homes. The department organized assistance programs, found lodging in apartments and hotels, and worked to get grants for furniture and clothing.
The firm also provided for employees who were stranded in the building Tuesday night either because they did not want to drive home in the dark or because they could not cross the bay. Two floors at the headquarters office were cleared and prepared for as much comfort and space as possible.
Growing hungry towards seven or eight that evening, the firm’s security personnel raided the cafeteria to help feed stranded employees. Schwab opted to hold off on the emergency food supplies, however, not only for rationing purposes, but also because “the food tastes awful and you don’t want to eat it unless you have to,” Lyons said.
Later that night, a senior executive offered to bring more supplies. Although the firm had radios, he brought battery-operated televisions (which have subsequently been stocked), flashlights, more food, and also personal hygiene supplies—toothbrushes, mouthwash, combs, etc.
The firm purchased similar personal hygiene supplies for the employees relocated to the nearby backup cold-site. Clothing, however, was more difficult to manage, but the firm did manage to provide Schwab T-shirts for the entire group. Rooms and meals were provided by the firm at a nearby hotel.
By making these extensive efforts, “the company gets a payback in the strength of its employees,” said Lyons. This comes in the form of loyal employees and a low turnover rate for Charles Schwab & Co., Inc.
The overwhelming success of the contingency plan speaks for itself, from the concerns of the employees to the overall recovery of the firm. Senior management, however, also benefited, especially from the plan’s conciseness and simplicity.
“We decided it would be too difficult to plan for every possible disaster: fire, flood, earthquake, etc.,” Lyons said. “So what we wanted to do was put together an informational tool, a checklist designed to help the executives THROUGH the disaster.” Senior management at Schwab reportedly found this approach extremely useful.
Lyons and his colleagues developed a basic checklist which, although it did not make decisions for them, helped executives assess and control damage. One strength of this approach was the elimination of unnecessary information. It contained plan reference numbers which, when accessed, provided specific details of the plan as needed. Executives felt comfortable with this approach.
Some of the items on this checklist provided information on customer service issues, such as how to pay customers and how to keep records during an emergency. These issues are of special concern since Schwab places such great emphasis on customer service.
A section of the plan also contained information on media and public relations issues, and, after examining executives’ needs, Schwab developed a daily briefing format.
In summary, Schwab designed a plan which recognized both employee needs and customer service as top priorities and allowed a freedom to improvise in meeting those priorities. Equally important, the plan proved extremely usable, containing a maximum of organization and a minimum of superfluous information.
Schwab has also considered canceling the next unannounced test. After all, since the plan proved successful during the actual emergency and also in the test ten days earlier, it seems unnecessary. Charles Schwab & Co., Inc. was, in fact the only brokerage firm in operation the day after the quake.
“Some of the branches down the peninsula were closed for a day,” said Lyons in closing, “but all of our bay area branches were open by 6:30 in the morning...you couldn’t even tell we had an emergency.”

Richard Newman is a staff writer for the Disaster Recovery Journal.

This article adapted from Vol. 3, No. 1, p. 16.

Disaster Recovery Worldİ 1999, and Disaster Recovery Journalİ 1999, are copyrighted by Systems Support, Inc. All rights reserved. Reproduction in whole or part is prohibited without the express written permission form Systems Support, Inc.