Disaster Recovery Journal

As you sit here reading this article, imagine your typical work day. You arrive at work and walk into your break room to grab a cup of coffee. You then move on to your work area, sit at your desk, organize your file folders, and check your inbox for any important documents. Now imagine the rest of your day without your telephone or computer system. How productive would you be? What would you do with your time if you were sitting in your chair at a desk without any electronic equipment?

For most of us, it would be difficult to operate under those circumstances. In disaster recovery, we typically find ourselves thinking about what we need to do to get our facility back in order first. Seldom are the first thoughts about equipment when discussing disaster recovery. However, in today’s average office or manufacturing facility, production would be at a halt without that equipment.

A case example involving a fire to a hospital operating room can illustrate this point. Recently, a hospital sustained a small fire in one of the operating rooms. While the heat level was relatively low, the soot and contamination levels were excessive throughout the room and associated medical equipment. Mix in the water from the breached sprinkler heads and the higher humidity from the HVAC not running, and all the ingredients are there to form highly corrosive hydrochloric acid (HCL). However, the thoughts of the hospital administration and the insurance carriers turned immediately to the building itself. The condition of the equipment was an afterthought. After three weeks of general facility restoration, the operating room was fully decontaminated and functional.

However, the equipment that was located in the operating room at the time of the fire still contained high levels of soot and smoke contamination. As such, the room was fully cleaned, but it had no equipment to complete any surgeries. Due to the lack of action on the equipment, the hospital was still unable to use the operating room and lost business continued. A six-figure revenue loss per day ensued as the room sat unused.

Discussions were held about replacing the equipment, but lead times were approximately six weeks and replacement costs were excessive. Further, an evaluation by an equipment loss consultant concluded that the equipment could be restored to pre-loss condition through professional decontamination. As such, an estimate was provided for equipment to be restored.

The estimate for decontamination was approximately 15 percent of the total replacement cost, and the lead time for completion was approximately three weeks, or half the time required for replacement. There was a cost savings in every sense. However, due to the fact that the equipment was an afterthought, business interruption costs continued to rise.

Had the equipment initially been thought of as an integral part of the overall facility, business interruption costs could have been limited, and the room could have been fully functional (equipment included) within the three-week time period allotted for the room restoration.

The above example is not uncommon. More times than not, equipment that has been contaminated in a loss is not seen as the most pressing issue, even when the equipment is required for high productivity of a business (i.e. a heart and lung machine in a hospital, or a printing press in a print shop). Further, the replacement costs and lead times associated with specialized equipment can be excessive. Equipment decontamination can reduce the overall impact of a loss when completed by a reputable restoration company that specializes in equipment.

In the following paragraphs, there are some points to consider following a contamination-related loss event. These include the effects of contamination on equipment, the validation of equipment restoration, industry accepted restoration techniques, recommended actions following a loss, and characteristics of a reputable equipment restoration firm.

The Effects of Contamination

After a disaster from fire and water, the chemicals deposited on the equipment can adversely affect electrical, electronic, and mechanical equipment. The effects of this contamination can vary widely, depending on the type of material or contaminant present, the amount or concentration of each respective contaminant, the environmental conditions present at the loss location, the equipment type, and the materials used in the construction of the equipment. The type of contamination deposited on the equipment is dependent upon the materials combusted. The contamination level varies on a myriad of factors, including, but not limited to, the location of the equipment in relation to the fire and to ventilation. Certain types of equipment are more susceptible to rapid deterioration than others. Specifically, systems using integrated circuits sustain irreparable damage more quickly than less sensitive mechanical devices. Additionally, the factors that affect the deterioration of equipment include temperature and humidity levels in the loss environment.

The US Nuclear Regulatory Office Description funded a study on the effects of combustion byproducts on electrical and electronic equipment. This study, titled, "Circuit Bridging of Components," described the types of failures that can occur as a result of contamination. In brief, the study listed the effects of fire on electrical and electronic equipment.

One of the most recognizable types of failure to equipment is corrosion. Corrosion, or oxidation, is a change in the properties of the metal caused by chemical reactions of the contamination and metal surfaces. Within electronics, this change in chemical properties also results in a change in the conductivity of the metal. These conductivity changes can result in spurious operation of the equipment as the corroded devices may not pass electrical and data impulses properly.

The mechanism of corrosion is dependent upon many factors. While the following items are an oversimplification, the factors that contribute to the amount and progression of corrosion including the type of materials utilized in the construction of the affected equipment, the type and concentration of corrosion, and the amount of moisture in the air. Depending on these factors, corrosion can progress rapidly and render some equipment irreparably damaged within 24 hours of exposure. Other equipment can sit for weeks without degrading if the contamination levels and environmental factors are favorable.

Aside from corrosion, certain types of equipment, such as disconnects in electrical switchgear and spindle motors in disk drives, to name just a few, can be damaged by mechanical binding if the contamination is not removed. Mechanical binding occurs because the contamination, whether corrosive or inert, acts as an abrasive and generates intense, localized heating. This heating reduces the viscosity of any lubricant present, which causes metal parts to rub and generates additional heat, resulting in a further reduction in viscosity. This cycle continues until the device binds and cannot move.

Another concern with corrosion is the overheating of electrical and electronic devices. The soot and contamination deposited from the fire may not be highly corrosive, but it still may prevent the sensitive micro-electronic components from properly dissipating heat. Over time, which may vary based on the thickness of the contamination and the types of components exposed, this continual buildup of heat can cause the device to fail. As an example, most processor-based equipment, including computers, telephone central office switches and CNC controls utilize heat sinks and fans on the processors to facilitate in the heat dissipation because the processors are so sensitive to overheating.

Intense, localized overheating can also occur to electro-mechanical equipment as a result of contamination. A good example of this is contactors in motor starters or switchgear. The contamination is not as conductive as the metal used in the contacts. The higher resistance of the contamination causes localized heating in the area. This heating causes the resistance to further increase in the area, which produces additional heat. Depending on the functionality of the device, the equipment may merely cease to function, or in the case of high voltage apparatus, may result in an explosion or fire.

Electrical shorts are another concern in fire losses. Not only may the contamination be conductive, the water used to extinguish the fire contains impurities that are also conductive. As a result, electricity may flow across areas or devices where it was not intended to flow. The end result may be an electrical short circuit and damage to the equipment.

Obscuration occurs to optical circuits. As the soot and contaminants from the fire are deposited on optical circuits, it disperses the beam of light used to transmit data. This dispersion or obscuration of the light can result in spurious operation as well as lost information.

Validation of Equipment Restoration

In an effort to understand not only the effects of combustion byproducts on electrical and electronic equipment, but also the likelihood of failure from this contamination, the U.S. Department of Energy undertook a study, which in part determined the probability of failure of equipment at various contamination levels. The results of their study, published in the DOE Fire Protection Handbook Volume II "Fire Effects on Electrical and Electronic Equipment," showed that the probability of failure of equipment increases exponentially with increasing contamination levels. The study provided scientific data regarding the contamination level below which the probability of failure is zero and also levels above which restoration is typically not cost effective. A graphical representation of their study is shown on the right.

To summarize the graph, the DOE study provides useful information in relation to equipment restoration. First, equipment restoration is typically not cost-effective above 500 μg/in² (micrograms per square inch) of chloride equivalent contamination. Factors affecting whether the equipment can be restored if contaminants are above this level include the type of equipment and the cost and availability of replacements. Another important result of the study is that the probability of failure is zero at contamination levels of 20 μg/in² or less. As such, in order for restoration to be reliable, the contamination level must be reduced to 20 μg/in² or less (chloride equivalent).

Restoration Techniques

To ensure that the restoration is successful, the equipment restoration company should choose the appropriate decontamination process based on several factors. The first factor is the type of components found inside the various pieces of equipment. Some components, such as optics and relays, do not respond well to aqueous cleaning. Equipment containing components of this nature, as an example, should be decontaminated using the modified aqueous method.

The second factor in determining the appropriate decontamination process is the degree of contamination. With exception to component specific requirements, more heavily contaminated equipment will require a more thorough cleaning using the full aqueous method. Equipment with less contamination can be cleaned using the modified aqueous method. When deciding on an equipment restoration company, you should discuss the methods used to ensure the proper care of the equipment. Below is just a brief explanation of some of the various industry accepted techniques used.

An aqueous process will be used on circuit boards and other components exhibiting medium to high levels of contamination. This same process is used by circuit board manufacturers in the final stages of production to remove any excess solder, flux, and other particulates.

This process will be employed in situations where the equipment cannot be removed from either the parent system or the location of where it currently is affixed. This would include but is not limited to items such as back planes, card cages, equipment racks, etc.

The "modified dry" decontamination process is utilized for equipment that sustained light contamination and/or where water cannot be used because it may damage specific, sensitive components. Examples of equipment on which the modified dry process should be used include optical circuit and relays.

Recommended Steps Following a Loss

In a contamination-related loss, there are several steps you can take to ensure that the equipment is being cared for properly. However, in each case, time is of the essence.

You want to be able to get your arms around this loss by qualifying and quantifying the damage to the contents.

The equipment needs to be stabilized by moving it to a humidity and temperature-controlled environment. If the equipment can’t be moved, use air-conditioning and/or dehumidifiers to lower the relative humidity. This will slow down the onset and progression of corrosion.

Ideally, humidity should be in a range between 45 percent and 55 percent and the temperature should be as cool as is practical. Humidity higher than this level will not sufficiently retard the corrosion process. Conversely, some equipment, mainly electronics, can be damaged by static electricity by the dry air when the relative humidity is less than 45 percent.

Prior to removing equipment, keep in mind that systems may be configured for certain locations or may contain privileged or confidential information. It may be very important to record and tag the equipment with the manufacturer, model number, serial number, and original location of each item prior to moving it.

v