Spring World 2018

Conference & Exhibit

Attend The #1 BC/DR Event!

Spring Journal

Volume 31, Issue 1

Full Contents Now Available!

Since the events of Sept. 11, 2001, many customers are now proposing a program of carrier diversity for their telecommunications services, especially their long distance services. This diversity program essentially splits their network between two or more carriers. The premise of this program is an assumption that their mission critical circuits, or their entire network, cannot fail simultaneously because of a network event in one carrier’s network. However, the customer is now burdened with the coordination between multiple telecommunications providers, which consumes assets, resources, and time.

In some instances the customers are indicating the program is being mandated by the U.S. Department of Homeland Security for certain mission critical circuits such as the banking and financial industries. Route diversity is achieved by selecting two separate facility routes that have no common spans between the originating central office and the terminating central office.

There are several levels of route diversity; there is the inter-office channel (IOC), the access channel, and the entrance facility into the customer premise. Each telecommunications carrier offers route diversity at these levels. Customers are now going for carrier diversity to further ensure that diversity is achieved. Carrier diversity is achieved by splitting your entire network or just the mission critical components between two or more telecommunications carriers.

 

On the surface this may appear to be a logical strategy. However, when you understand how the telecommunications industry operates, you will realize that this strategy severely increases the customer’s risk for the mission critical components of their network. To illuminate the severe increased risk to the mission critical components created by such a program requires an understanding of the routing programs and policies of the long distance carriers.

The most important policy is that long distance carriers do not share engineering databases indicating where their cables are buried or where their circuits are routed. Therefore, there is no way that one provider can know how the other carrier will route the same circuit. If you have two mission critical circuits and you are using a different carrier to provision each of them, there is a very strong chance that those two circuits will be riding the same cable routes across part of the country.

There are several reasons for this phenomenon. First, and most importantly, all of the carriers share common rights of way. One of the highest costs involved with installing a telecommunications cable is obtaining the rights of way from the various property owners to traverse their land. When you consider the vast mileage that is traversed by the mesh of nationwide cables you begin to understand the complexity of negotiating rights of way. The most common rights of way that are utilized are the transcontinental network of railroad lines and the nationwide highway system. The rights of way already exist and the required equipment, such as specially equipped railroad cars and buried conduits, already exist. What has happened here is that all of the telecommunications carriers have deployed their cables along the existing railroad routes and along the vast network of highways and roadways. Even if the cables are buried on opposite sides of the railroad tracks, or on opposite sides of the highway, they still traverse the same network of bridges and tunnels. Any collapse or damage to a bridge, or an explosion or fire in a tunnel, could cause a major network event to every telecommunications carrier utilizing that right of way. Any train derailment also runs the same risk because it is extremely easy for a fully loaded freight train to gouge out a four-foot deep trench. Also, the hazardous chemicals carried by the railroads could greatly delay the time it takes to repair the damaged cables and could complicate the network impact. While this is an extremely serious risk that the telecommunications carriers accept, most customers are not aware of the magnitude of the impact because telecommunications providers have developed automated reroute systems that immediately restore the customer’s communications capability.

The long distance communications providers also participate in a program of dark fiber swaps. Dark fibers are spare, unlit fibers in an existing cable. This fiber swap involves one carrier trading spare dark fibers in one geographic area for spare dark fibers from the other carrier in another geographic area where they have a customer demand. Essentially this places both carriers on separate fibers within the same cable. Farmers and construction backhoes do not discriminate between particular fibers, they just cut the entire cable. If your mission critical circuits were diversified between carriers, but routed on the same dark fiber swap, then in the event of a failure on that cable your mission critical functions have just ceased to operate. These dark fiber swap cables are inventoried as if they were an installed component of the carrier’s network. Since the various carriers do not share routing engineering with each other it is impossible to know if both circuits are being routed on the shared cable.

The long distance communications providers also participate in leased facilities programs with each other. While these leases tend to be expensive they are still cheaper than installing a new cable. This provides temporary capacity to an area until a permanent solution can be developed. Also, the major difference between a fiber swap and a fiber lease, besides ownership, is the fact that with a fiber swap the telecommunications carriers receive the cable routing information for their purposes, with a lease the telecommunications carriers do not receive the exact cable routing information. One of the telecommunications carriers that I know of does not use leased facilities on circuits that have been ordered with route diversity because of the lack of exact routing information.

The same telecommunications carrier also runs several automated systems that perform network grooming and facility optimization. These systems analyze the circuits and ensure that they are routed on the most efficient facility path with the minimal amount of mileage. These programs run every night and automatically move the circuits to new facility paths. This process is completely invisible to the customer. If circuits are ordered with special features for diversity then the provisioning and facility systems will set indicators that will provide an automated warning that these circuits have been modified and now have potential diversity issues. Corrective action will then be taken to clear any diversity violations.

Telecommunications carriers also perform fiber clearing in an effort to obtain requested customer capacity or to perform necessary cable maintenance. Without knowing how the circuits of the other carriers are routed there is no way to guarantee that the mission critical circuits will remain diverse, hence, all mission critical circuits should be provided by one carrier.

Therefore, while these events may seem significant to the customer, they happen on a frequent basis. The reason the customer is not impacted is because of the combined efforts of the automated reroute systems and the network managers in the network operations centers. Telecommunications carriers can control their own network and provide seamless communications to their customers. This capability is severely impacted when a customer decides to split its network between multiple providers or carriers. The occurrence of cable cuts and equipment failures are measured in weeks between failures, but the associated risks can be minimized or eliminated by the use of a single carrier. The occurrence of a complete network failure by a single carrier is extremely rare; this type of event is measured in tens of years between failures.

Customers using carrier diversity for their mission critical network components are at a greater risk for a cable cut that affects both carriers simultaneously, than they are for a rare event such as a total network failure of a single carrier. It also places all of the risk and coordination squarely in the lap of the customer as opposed to the communications carriers that fully understand the inner workings of the industry. Why should a company invest in a consultant to perform continuous circuit routing comparisons from multiple carriers, provided they can get the data required, as opposed to letting the telecommunications carriers do it at minimal charge because they can control the routing if they have both circuits? Even if the multiple carriers that are providing the communications services meet with the customer at the same time to display their routing and protection, that can all change within 24 hours due to network optimization, normal maintenance functions, and automated reroutes. There is no way that the customer, or any consultant hired by the customer, can guarantee that the circuits and facilities are completely diversified and the customer’s business is free of risk. Unfortunately, just the opposite is true; the risk to the customer’s critical business has been significantly increased. Therefore, only the use of a single telecommunications provider for the mission critical components can ensure true diversity and reliability for the protection of the customer’s network.


James K. Crosson, MBCP, MBCI, is a technology consultant with AT&T Labs and has more than 33 years of network service. Crosson is responsible for analyzing the mission critical functions and network components of the top 100 AT&T customers. Crosson is a DRI International certification commissioner and is the founder, president, and director of membership for the Liberty Valley and Mid Penn chapters of the Association of Contingency Planners.