|
TELECOMMUNICATIONS
Central
Office Disaster Recovery: The Best-Kept Secret
By MICHAEL SMITH
Since the Sept. 11
tragedy, disaster recovery has become a household word, there are even
advertisements in Rolling Stone magazine. This has not been the case
over the past 20 years. Early disaster recovery plans, initially developed
for mainframe computers, called for back-up tapes to be rushed to off-site
locations and loaded onto waiting computers provided by outside companies.
The process plan was to be initiated in the event of a computer failure
or damage to a building housing the computer. In the 1980s, with extensive
damage being caused by hurricanes, earthquakes, and other natural disasters,
companies began thinking about preparing themselves for an event that
might affect their computers and communications lines.
Two things happened in the late 1980s and early 1990s that changed the
way big companies prepared for the worst. On May 8, 1987, there was
a fire in a telephone company switching office, known as a central office,
in Hinsdale, Ill. That particular central office provided service to
35,000 customers, and served as a hub for 50 other central offices to
connect to long distance providers. Fiber optics and advances in technology
had made it tempting for telephone companies to “put all their
eggs in one basket,” taking telecom technology to the point where
one location, in this case Hinsdale, served as a gateway to the long
distance network for hundreds of thousands of customers.
In addition, the Hinsdale fire destroyed the link between Federal Aviation
Administration air traffic controllers at O’Hare and other FAA
control centers, as well as severing vital data circuits for large numbers
of commercial companies. There had never been a disaster of such magnitude
in the history of telecommunications. At the time even the Defense Department
expressed concern over the vulnerability of the telephone network. It
took four weeks to restore service to all customers affected by the
disaster at Hinsdale. At the same time new banking regulations were
being planned by the comptroller of the currency that required banks
to back up all of their communications lines. Those new regulations
directed the board of directors of banks and securities companies be
held personally liable for disaster planning non-compliance. Banking
Circular reinforced a requirement for financial institutions to promulgate
prevention and reaction plans to counter the disastrous financial impact
of severe telecommunications outages, and the conceivable effect of
such events on the national economy.
The financial community needed a solution to their dilemma. With a compliance
deadline fast approaching, New York Telephone (now Verizon) was pressured
by its Manhattan customers to come up with a quick solution. The phone
company didn’t have a suitable service to offer. At the same time,
a major magazine publisher in Manhattan was concerned about protecting
its service (they already had been utilizing fiber optic services for
several years, and were the first company to use alternate path protection).
That publisher approached their telephone company representative, Bob
Barker, in April of 1988 requesting a service that would insure continuity
in the event of a central office failure.
The solution came about in an interesting way. While interests of the
financial community centered on making outgoing telephone calls (e.g.,
brokers calling clients), the publisher’s priority was focused
on receiving incoming telephone calls (e.g., from reporters calling
in stories). It turned out that the request from the publisher to route
incoming calls around their central office was the key to solving the
problem.
At the time, central office boundaries were sacrosanct when it came
to local switched services. They had never been crossed, each central
office serving a fixed geographic area. Fiber optic cables had been
in use since the early 1980s, primarily for interoffice trunks. Although
not in abundance, fiber optic cables were becoming increasingly available
from central offices to local business locations.
Barker was able to convince his employer, the local phone company, to
cross the local central office boundary line with fiber optic cable
and extend it to the publisher’s building directly from a neighboring
central office.
Helping the situation was the close proximity of central offices in
New York City, and the availability of fiber optic cables in the streets
of that city. The publisher’s PBX was a Northern Telecom SL-100,
which was the customer equivalent of the telephone company’s DMS-100,
typically used as a central office switch in the phone company’s
network. This allowed the phone company to designate the publisher’s
SL-100 as an end office, or local central office, in its own right.
In addition, New York Telephone had an entire NXX code (9,999 numbers,
normally referred to as “a block of numbers”) assigned exclusively
to the publisher. This was significant, because a tandem office only
recognizes the first three digits of a phone call (ex. 123-1234), when
routing incoming telephone calls to the local central office that hosts
those numbers. The local central office then directs an incoming call
to its final destination, according to the last four digits of the phone
number (ex. 123-1234).
The next thing done was the programming of tandem central offices to
send incoming calls directly to the publisher over existing fiber routes
that had been established. The tandem was also programmed to send calls
the regular way it had before the diverse route was added. Tandem offices
are used to route calls to local central offices, to pass calls along
to other regional areas, and to send calls to a subscriber’s designated
long distance provider as well. In addition to the diverse path, a neighboring
central office provided lines for the publisher’s outgoing service
as well. The diagrams on this page show the normal service delivery
method (figure 1), and delivery using wire center diversity (figure
2).
At the time, the rerouting of calls could only be done by a tandem office.
Without Barker’s rerouting capability, disaster recovery for communications
lines would have been delayed for years. The rerouting of numbers is
only possible because of a tandem’s inherent capability to route
calls to large (9,999) blocks of telephone numbers. In the World Trade
Center disaster, the NXX codes housed in the central offices affected
by the devastation were rerouted to alternate central offices where
arrangements had been made to send the calls to other locations.
The solution provided to the publisher, under what was termed a limited
service offering, became the precursor for new tariffed services for
New York Telephone (Verizon). The umbrella name for such services is
Alternate Serving Wire Center (ASWC). These services include digital
data line services such as SIDN, Enterprise Service, DS-1s, DS-3s and
SONET Rings. In addition, switched services are backed up by the service
tariffed as Flexpath Disaster Recovery Service.
Because of the proprietary nature of disaster recovery (disaster recovery
should protect against sabotage, too), and the banking and securities
firms that use the service, little is published about it besides references
to rerouting. In fact, even phone company personnel not directly involved
with it know little or nothing about it.
It is clear, however, that every bank and security company in New York
City, and the rest of the country as well, are using this service.
In addition, this is a rare example of a service developed by an employee
of the phone company, instead of being invented by a telecommunications
equipment vendor. This critical disaster prevention and recovery service
is not only simple in its concept and execution, it has no doubt generated
incremental revenues of around $1 billion since its inception, based
on a report from telecommunications expert John Donovan, president of
Telecom Visions, Inc. of Garden City, N.Y. It’s one of the best-kept
secrets in the telecommunications industry.
Michael Smith is an engineering
consultant with more than 20 years experience in the electronics and
telecommunications industry. His work includes network and system level
architecture and design for companies including IBM, Siemens, General
Electric, and Lockheed Martin. He currently resides in Clearwater, Fla.,
and can be reached at msmit1@tampabay.rr.com.
To comment on this article, go
to 1502-04 at www.drj.com/feedback.
|
