Spring World 2018

Conference & Exhibit

Attend The #1 BC/DR Event!

Spring Journal

Volume 31, Issue 1

Full Contents Now Available!

Since the Sept. 11 tragedy, disaster recovery has become a household word, there are even advertisements in Rolling Stone magazine. This has not been the case over the past 20 years.

Early disaster recovery plans, initially developed for mainframe computers, called for back-up tapes to be rushed to off-site locations and loaded onto waiting computers provided by outside companies. The process plan was to be initiated in the event of a computer failure or damage to a building housing the computer. In the 1980s, with extensive damage being caused by hurricanes, earthquakes, and other natural disasters, companies began thinking about preparing themselves for an event that might affect their computers and communications lines.

Two things happened in the late 1980s and early 1990s that changed the way big companies prepared for the worst. On May 8, 1987, there was a fire in a telephone company switching office, known as a central office, in Hinsdale, Ill. That particular central office provided service to 35,000 customers, and served as a hub for 50 other central offices to connect to long distance providers. Fiber optics and advances in technology had made it tempting for telephone companies to “put all their eggs in one basket,” taking telecom technology to the point where one location, in this case Hinsdale, served as a gateway to the long distance network for hundreds of thousands of customers.

In addition, the Hinsdale fire destroyed the link between Federal Aviation Administration air traffic controllers at O’Hare and other FAA control centers, as well as severing vital data circuits for large numbers of commercial companies. There had never been a disaster of such magnitude in the history of telecommunications. At the time even the Defense Department expressed concern over the vulnerability of the telephone network. It took four weeks to restore service to all customers affected by the disaster at Hinsdale. At the same time new banking regulations were being planned by the comptroller of the currency that required banks to back up all of their communications lines. Those new regulations directed the board of directors of banks and securities companies be held personally liable for disaster planning non-compliance. Banking Circular reinforced a requirement for financial institutions to promulgate prevention and reaction plans to counter the disastrous financial impact of severe telecommunications outages, and the conceivable effect of such events on the national economy.

The financial community needed a solution to their dilemma. With a compliance deadline fast approaching, New York Telephone (now Verizon) was pressured by its Manhattan customers to come up with a quick solution. The phone company didn’t have a suitable service to offer. At the same time, a major magazine publisher in Manhattan was concerned about protecting its service (they already had been utilizing fiber optic services for several years, and were the first company to use alternate path protection). That publisher approached their telephone company representative, Bob Barker, in April of 1988 requesting a service that would insure continuity in the event of a central office failure.

The solution came about in an interesting way. While interests of the financial community centered on making outgoing telephone calls (e.g., brokers calling clients), the publisher’s priority was focused on receiving incoming telephone calls (e.g., from reporters calling in stories). It turned out that the request from the publisher to route incoming calls around their central office was the key to solving the problem.

At the time, central office boundaries were sacrosanct when it came to local switched services. They had never been crossed, each central office serving a fixed geographic area. Fiber optic cables had been in use since the early 1980s, primarily for interoffice trunks. Although not in abundance, fiber optic cables were becoming increasingly available from central offices to local business locations.

Barker was able to convince his employer, the local phone company, to cross the local central office boundary line with fiber optic cable and extend it to the publisher’s building directly from a neighboring central office.

Helping the situation was the close proximity of central offices in New York City, and the availability of fiber optic cables in the streets of that city. The publisher’s PBX was a Northern Telecom SL-100, which was the customer equivalent of the telephone company’s DMS-100, typically used as a central office switch in the phone company’s network. This allowed the phone company to designate the publisher’s SL-100 as an end office, or local central office, in its own right. In addition, New York Telephone had an entire NXX code (9,999 numbers, normally referred to as “a block of numbers”) assigned exclusively to the publisher. This was significant, because a tandem office only recognizes the first three digits of a phone call (ex. 123-1234), when routing incoming telephone calls to the local central office that hosts those numbers. The local central office then directs an incoming call to its final destination, according to the last four digits of the phone number (ex. 123-1234).

The next thing done was the programming of tandem central offices to send incoming calls directly to the publisher over existing fiber routes that had been established. The tandem was also programmed to send calls the regular way it had before the diverse route was added. Tandem offices are used to route calls to local central offices, to pass calls along to other regional areas, and to send calls to a subscriber’s designated long distance provider as well. In addition to the diverse path, a neighboring central office provided lines for the publisher’s outgoing service as well. The diagrams on this page show the normal service delivery method (figure 1), and delivery using wire center diversity (figure 2).



 


At the time, the rerouting of calls could only be done by a tandem office. Without Barker’s rerouting capability, disaster recovery for communications lines would have been delayed for years. The rerouting of numbers is only possible because of a tandem’s inherent capability to route calls to large (9,999) blocks of telephone numbers. In the World Trade Center disaster, the NXX codes housed in the central offices affected by the devastation were rerouted to alternate central offices where arrangements had been made to send the calls to other locations.

The solution provided to the publisher, under what was termed a limited service offering, became the precursor for new tariffed services for New York Telephone (Verizon). The umbrella name for such services is Alternate Serving Wire Center (ASWC). These services include digital data line services such as SIDN, Enterprise Service, DS-1s, DS-3s and SONET Rings. In addition, switched services are backed up by the service tariffed as Flexpath Disaster Recovery Service.

Because of the proprietary nature of disaster recovery (disaster recovery should protect against sabotage, too), and the banking and securities firms that use the service, little is published about it besides references to rerouting. In fact, even phone company personnel not directly involved with it know little or nothing about it.

It is clear, however, that every bank and security company in New York City, and the rest of the country as well, are using this service.

In addition, this is a rare example of a service developed by an employee of the phone company, instead of being invented by a telecommunications equipment vendor. This critical disaster prevention and recovery service is not only simple in its concept and execution, it has no doubt generated incremental revenues of around $1 billion since its inception, based on a report from telecommunications expert John Donovan, president of Telecom Visions, Inc. of Garden City, N.Y. It’s one of the best-kept secrets in the telecommunications industry.


Michael Smith is an engineering consultant with more than 20 years experience in the electronics and telecommunications industry. His work includes network and system level architecture and design for companies including IBM, Siemens, General Electric, and Lockheed Martin. He currently resides in Clearwater, Fla., and can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it..