Each business is impacted differently by a power failure. Some businesses, such as stock brokerages, rely on computers for the exchange of electronic information. Electrical power and telecommunications are critical to their business. Other businesses, manufacture products to be sold. Factory production lines do not operate without electrical power. Imagine the impact of loss of electricity on a hospital. Thus, the loss of electric power creates a unique set of problems for each business.
Much attention has been given to the subject of disaster recovery for data processing centers. The concept of using 'mirror' or 'back-up' sites is not new. The principle is simple. If the primary data processing center fails, the mirror, or back-up data processing center keeps the information flowing. Well planned, this form of disaster recovery or disaster prevention, is quite effective. Many firms offer consulting or data processing services for such a contingency.
What about businesses that can't move their processes to another location (such as a manufacturing plant or a hospital)? If utility power is unavailable, another power source must be found to keep the business open. The focus of this article is on back-up power systems and some of the problems a business continuity professional may face when planning for extended, regional power failures.
The most common back-up power source is the engine-generator. Many large facilities have some sort of engine-generator as a back-up power source. The questions the business continuity professional must consider are: 1) Does the engine-generator back-up all critical functions as they relate to the business 2) Will the engine-generator function for extended periods of time without failing? 3) Are there any other factors that might affect engine-generator performance?
Some businesses elect not to have a engine-generator permanently installed on site. Instead, these businesses might contract with a local engine-generator supplier or disaster recovery firm, that can quickly transport a portable engine-generator to the facility. In most cases it is necessary to have a written agreement before the disaster occurs. The business continuity professional must ensure the correct engine-generator is specified for the building loads, and how the disaster plan is to be implemented.
Each day, thousands of businesses have short term power failures (a few minutes to a few hours) and run on their back-up engine-generators to power their facility when utility power fails. The interesting thing about power failures is that you can not determine how long they will last. When the lights turned out in millions of U.S. and Canadian businesses and homes in January 1998, most people did not suspect that the lights would not return for up to four weeks. Those with good disaster recovery programs suffered far less than those who had not planned for such an event. Listed below are a few of the more common reported problems when back-up systems and disaster recovery plans fail.
The back-up engine-generator would not operate. Utility power in North America is very reliable. Some facilities almost never have a power failure and consequently do not have to rely on their engine-generator. The engine-generator is forgotten and not maintained and tested routinely. As such, it is not ready to perform during an emergency.
The back-up engine-generator shut down after several hours of operation. Many engine-generators are built for 'intermittent duty', that is a few hours at a time. After a few hours service, they must be shut down and serviced. When specifying a back-up engine-generator it is imperative to know the 'duty cycle' of the device. The maximum time between service intervals must not be exceeded. Business continuity planners should recommend a redundant engine-generator scheme. This involves installing two engine-generators to do the work of one. The engine-generators normally operate one at a time. However, they can be paralleled so that both are on-line before one is shut down for service. The building service is not interrupted while switching between the two engine-generators.
The fuel runs out. Most engine-generators are powered by diesel fuel or natural gas. Natural gas is a very reliable utility, but lines may be compromised during an earthquake. Plans must be made for an alternative source of natural gas should the utility gas supply fail. Businesses which have diesel fuel powered engine-generators installed probably don't have more than three days of fuel on site. It is imperative to make sure that a reliable fuel vendor is available during a regional power emergency. If the fuel vendor does not have an adequate disaster recovery plan, then they may be temporarily out of business when you need them most. No portable engine-generators were available. Your company signed an agreement with a vendor to provide an engine-generator in the event of a long-term power failure. Unfortunately, the vendor signed many such contracts and has more customers than engine-generators. Where do you sit in the pecking order? Is it first call - first served? If millions of businesses and homes lose power for an extended period of time, it might be difficult to get the back-up generator you were counting on. Perhaps the vendor you contracted with did not have a disaster recovery plan and is temporarily out of business. This is the best argument for having a permanently installed back-up system, as opposed to contracting a portable system.
Difficulty installing the portable engine-generator. If the business continuity plan includes bringing a portable engine-generator to the facility, the plan must include the means of quick and simple installation. An area must be set aside for placement of the portable engine-generator. Cabling will have to be run from the engine-generator to the main power switchgear. This might not be easy since the power cables are large and have to be run through doorways, up or down stairways, possibly long distances. A better approach is to have wiring in place that connects the main power switchgear to a weatherproof electrical junction box located on the outside of the building next to the area reserved for the portable engine-generator. This allows for quick and simple connection of the engine-generator power cables.
Complaints that engine-generator power is not very good. The quality of engine-generator power varies greatly. Severe voltage and frequency variations can occur, causing building loads to shut down or fail. Contributing factors include the quality of the engine-generator, the type of voltage regulator and governor installed, how heavily loaded the engine-generator is, and what types of loads are being served. Placing a true on-line uninterruptible power system (UPS) between the main panel switchboard and building loads will protect sensitive loads from voltage and frequency fluctuations. The UPS will provide continuous protection and power to the loads during short-term power failures as well.
Each business is unique. No two businesses have the same requirements for electrical power. The business continuity professional must sort out the power requirement needs for that particular business. Even the best plan can have holes in it. Try to consider all possible scenarios that may occur during an extended power failure. Remember that the vendors and contractors that you are dealing with may be temporarily out of business during an electric power or telecommunications outage and may not be available to provide the goods and services that your business requires during a disaster.
Plan for the worst and hope for the best.
David W. Goodrich has held numerous positions with the Liebert Corporation since the mid-eighties. He will speak at the DRJ Fall World 1998 Conference in Orlando, FL