Making The Connection

By Leonard Lee

For many businesses, from airline reservation centers to stockbrokers, the telephone is their lifeline. But businesses are quickly discovering just how fragile that lifeline can be.
The dependence of the nation’s businesses on reliable telecommunications is tremendous. In New York City alone, with its myriad of financial markets and corporate headquarters, an estimated one trillion dollars in daily financial transactions depends on telecommunications networks. Sixty-three percent of New York’s non-agricultural jobs are in telecommunications-intensive industries.
But three times in the last two years, New York has experienced the loss of the greater part of its telephone service. And other metropolitan areas have suffered similar outages.
On September 17, 1991, a broken rectifier in a Manhattan switching center shut down New York City’s AT&T long distance service for six hours. The outage occurred late in the afternoon, meaning most businesses were unaffected. But it cut off contact between the region’s several air traffic control centers. The result was lengthy delays at New York’s three major airports. The resulting backups delayed air traffic throughout the country, disrupting travel for thousands of passengers.
Another recent phone outage was national in scope. On January 15, 1990, AT&T lost most of its long-distance service nationwide for eight hours. Businesses ranging from telemarketers to stock brokers to travel reservation centers lost hundreds of millions of dollars in business. Only 40 percent of long distance calls placed that day went through.
In 1991, telephone outages affected major metropolitan areas in the U.S. on 11 different occasions. (See figure 1). The outages lasted anywhere from fifteen minutes to eight hours. Customers were either unable to complete calls within their local dialing zone, or were unable to make long-distance calls. The effect on businesses in the affected areas was devastating.
Companies lost astonishing amounts of business in very short periods of time. A stock broker in Washington, D.C. estimates his office alone lost $25,000 in business when a computer switching problem knocked out phone service to the District of Columbia, Baltimore and parts of Maryland, Virginia and Delaware for six hours in June. In the nationwide AT&T disruption in January 1990, American Airlines’ reservation center in Tulsa, Oklahoma lost an estimated 200,000 calls.
Alarmingly, such disruptions of telephone service are becoming frighteningly common in an era of computerized telecommunications. While new technologies have brought a wider choice of vendors and services, such as teleconferencing, computer data transmission and international direct dialing, they have also introduced an era of new hazards.
Failures that once would have been minor in nature now induce widespread disruptions of service. The computerization of telephone switching centers means that where five years ago an AT&T switching center handled about 180,000 calls per hour, new computerized switches handle 700,000 calls an hour. The loss of a single switch (AT&T has 114 around the country) now means a much wider disruption of service. Likewise, modern fiber optics carry 10,000 times more calls than the old copper cables they replaced. An accidental cut of a single fiber optics line can cut off entire metropolitan areas, as happened in New York and Washington, D.C. in January and March of 1991. In the last four years, this has happened at least 11 times.
A National Research Council report in 1989 warned, “A single switch (or fiber optics line) may support communications for many tens of thousands of subscribers in multiple communities. This trend increases the potential for catastrophic disruption. The nation’s telecommunications and information networks are becoming more vulnerable to serious interruptions of services.” (“Growing Vulnerability of the Public Switched Networks,” National Research Council, 1989)
Such problems are not limited to AT&T either. Similar outages have affected both MCI and Sprint, AT&T’s main competitors. But because AT&T carries approximately 70 percent of the nation’s long-distance calls, any difficulties are inherently more serious than those affecting other carriers.
Fortunately, there are steps businesses can take to protect themselves against disruption of telephone service. Some solutions are not suitable for small businesses. And no defense is fool-proof against all situations. But preparation and foresight can minimize the danger.
Geographic diversification is the best defense against the risks of phone outages. Relying on a single point for your telecommunications services can be an Achilles Heel. Your business can be highly vulnerable to a single well-placed disruption. Florists Transworld Delivery found this out, much to their dismay in May 1989. A fire at an AT&T switching center in Hinsdale, Illinois knocked out the only “gateway” for calls into FTD’s order processing department. Twelve thousand florists around the country could no longer communicate with FTD. Thousands of orders of flowers went undelivered.
Holiday Inns could easily have suffered a similar fate. Its regional reservations center, also located near Hinsdale, normally handles about 35,000 incoming calls for reservations a day. But Holiday Inns was able to transfer its call traffic to another reservation center in North Carolina.
While the advent of computer phone switching centers introduces new risks of widespread service disruption, it also offers a partial salvation. The use of new so-called common channel signalling by long distance carriers permits instant switching of incoming calls to alternate locations in the event of a local phone outage.
In the Hinsdale outage, full phone service to some businesses was not restored until two months later. Such a lengthy disruption could easily have a disastrous impact on businesses.
There are precautions businesses can and should take to minimize the impact of any disruption of telephone service.

Access codes.

At minimum, be prepared to use other long distances services for out-going calls. Whichever long distance carrier you use, other competing services can be accessed.
to access AT&T, dial 10288 + number
to access MCI, dial 10222 + number
to access Sprint, dial 10333 + number

Have more than one long distance carrier

Contracting with multiple long distance carriers can minimize the disruption caused by any single long-distance outage. Many businesses split their telephone lines among two or more different carriers. Loss of a single carrier may reduce call capacity, but you will then retain at least some incoming and outgoing call capabilities.
Utilizing more than one long-distance carrier may, however, increase costs somewhat, because of the loss of volume discounts. One additional note of caution: Make sure your long-distance carriers are actually using separate lines. Carriers often sell capacity to one another, so your back-up carrier may actually be using the same physical line as your primary carrier. Ask your carriers to specify the physical routing of your lines to insure you are buying true redundancy.

Have a back-up plan for loss of phone service

Anticipate the loss of both local and long distance, incoming and outgoing phone service, and plan appropriate contingencies. Contract with courier services, both local and inter-city to transport vital documents and written communications. (And make certain you have a way to contact the courier other than by phone) In the event of loss of local phone service, bicycle or motorized couriers may be the only way to communicate between offices in the same city.
Inter-city, some large businesses already possess alternate modes of communications. Satellite teleconferencing or in-house television networks can be used to maintain contact with branch offices, and emergency contingency plans for such should always be kept in place.

Wire directly into long-distance networks

Some large businesses now bypass local phone companies completely. Their phone systems are directly linked to long-distance carriers by fiber optic cable or microwave relays. This reduces the risks of local phone disruptions interrupting long-distance connections.

Geographic diversity

If you are dependent on incoming calls for your business, never have the calls funneled into one location. Maintain more than one center for handling incoming calls. Contract with you 800-number or regular long distance carrier to insure calls will automatically be diverted to the other location in the event of a phone outage.

Build redundant local networks

If communications or data transfer between a central office and branches is critical, build redundant means of communications. In New York City, for example, companies such as Teleport Communications and Metropolitan Fiber Systems provide private telecommunications networks that can offer communications links through their fiber optics lines. Make sure your back-up lines are truly redundant, to the point of making sure they take different physical paths through the city and even entering your building.
The watchword is be prepared. Phone outages are occurring with alarming frequency lately. In today’s high-tech telecommunications environment, it is no longer a question of if phone outages will occur, but merely when.
Such failures can be devastating to businesses. But a few precautions and adequate preparation can lessen the impact and minimize the dangers. Proper disaster management techniques can prevent your business from getting “hung up” the next time the phones go down.

Leonard Lee is the author of “The Day the Phones Stopped,” a book dealing with problems caused by computer errors, including telecommunications outages. He is a news reporter with KSTP-TV in St. Paul, Minnesota.

This article adapted from Vol. 5 #1.

Disaster Recovery Worldİ 1999, and Disaster Recovery Journalİ 1999, are copyrighted by Systems Support, Inc. All rights reserved. Reproduction in whole or part is prohibited without the express written permission form Systems Support, Inc.