To Avoid Millions in Lost Production During Utility Power Outage
The Volvo GM Heavy Truck Plant in Orrville, Ohio, avoided millions of dollars in lost production by quickly securing and hooking up 5 megawatts (MW) worth of rental generator sets to power the facility for nearly two days during a recent, unexpected electrical utility outage. At the same time, the plant developed a solid working model for using temporary power during future outages.
The 475,000-square-foot plant, owned by Volvo GM Heavy Truck Corporation, manufactures upwards of 36 custom-designed heavy trucks a day that include automobile haulers, dump trucks and cement mixers. During the past four years, the Orrville Assembly Plant has more than tripled its production to more than 11,000 trucks annually to keep pace with record sales years. The facility's 1,004 employees routinely work one eight or nine-hour daytime production shift as many as six days a week, with two daily maintenance shifts scheduled to restock materials on the production line.
Utility Fire Halts Operations
Production at the Orrville facility has never been stopped by an unscheduled interruption in the plant's seven-year history. That situation changed, however, after the plant lost utility power at 9 p.m. on Tuesday, Feb. 28, when a fire broke out at the Orrville Municipal Utilities power facility, the local utility powering the plant. Fortunately, only a small maintenance crew was working in the plant at the time, and production was not running. The problem at the utility involved a general circuit breaker failure on a 13,800-volt (V) electrical bus that destroyed three adjacent breakers, electrical and control wiring, and thousands of feet of transmission and distribution lines. The damaged wiring dictated the need to disconnect the utility's turbine generators, leaving most of the community's homes and businesses without power for as long as five days.
At the Volvo plant, four dedicated substations are tied into the city's utility grid via high-voltage feeder cables. Substation transformers step down incoming power from 13,800V to the required 480V. Power is then fed to the plant's 800-amp electrical buses, which tap off circuit breakers to distribute power throughout the facility.
When the utility fire began, transmission lines feeding the plant were immediately shut down, as were distribution lines from another utility that served as the plant's only emergency backup power source. To restore power to the operation, the plant's crisis management response team met at 10 p.m. in the plant's offices, where a 36 kilowatt (kW) emergency generator provided partial lighting and telephone service. Although utility officials were unavailable to discuss the outage immediately, the response team determined from radio reports that power was likely to be off-line for days. It was immediately decided that temporary power from rental gen sets would be the only alternative to quickly restore power to the plant.
The goal was to restore power before the next production shift because unscheduled shutdowns are not acceptable to Volvo: Unplanned downtime is estimated to cost approximately $2 million a day in lost production. Additionally, the plant would need to break its guaranteed delivery dates, leading to lost sales and possibly hindering future business. And because the power outage occurred during a particularly cold winter, the lack of available heat threatened to freeze the plant's sprinkler system, posing a fire hazard.
Power Needs Assessed
The response team spent the night of Tuesday, Feb. 28, addressing life-safety needs and guarding the plant against possible fire hazards. The work included closing paint cans and paint ovens to prevent fumes from escaping, shutting off equipment and checking for loose wiring. After safety needs were addressed, water booster pumps were shut off to bolster the city's water supply, which was needed to fight the utility blaze.
By 7 a.m. on Wednesday, the response team decided the top priority was to power the plant's critical loads, including lighting, HVAC systems and a computer database that performs inventory control functions for Volvo's entire North American truck operations.
The plant's large, U-shaped production line and smaller feeder lines also needed power, the bulk of which is used to drive air compressors that cool paint booths to keep solvent from thickening and to run the conveyor system for the plant's assembly lines.
In addition, it was necessary to relay power from the plant to a nearby cab assembly facility, owned by a local truck-parts manufacturer that furnishes truck bodies to the plant. The cab assembly plant, which was also without power, was tied into the main plant's utility distribution system.
After quickly analyzing past utility statements, the response team determined that satisfying the Volvo plant's peak electrical demand, which occurs during production shifts from 6 a.m. to 3 p.m., required 3.5 MW of power. The plant has monthly consumption of more than 800,000 kilowatt-hours (kW-hrs) in the winter, when gas heat is generated. In addition, the nearby cab assembly plant also needed an additional 1.75 MW of power. In total, 5.25 MW of temporary power was needed.
The response team immediately, contacted a local supplier on Wednesday, to secure the needed rental gen sets. The temporary power equipment was to be delivered and installed by 5 a.m. the next day to allow the next production shift to resume work at 6 a.m. The regularly scheduled production shift on Wednesday was called off.
Temporary Power Plan Executed
By 9 a.m. on Wednesday, the response team finalized its temporary power restoration plan calling for gen sets to tap into utility lines connected to the plant's substations. The response team originally considered connecting directly to the substation transformers but found the solution impractical because the rental units and auxiliary equipment would have needed to be placed near plant doorways, where they posed a safety risk and blocked plant access.
As such, a decision was made to connect the gen sets via transformers to stress cones on an existing utility pole in the plant's parking lot and feed power from utility distribution lines to the four substations. This option was selected because it posed no obstacles for plant operations and reduced installation time.
However, it dictated the need for the temporary power equipment to match the voltage and amperage of the utility lines.
By noon Wednesday, the first of three utility-grade power modules, each capable of producing 1,750 kW at 480V, arrived at the plant in mobile, 8 X 40 foot trailerized units. The other two power modules arrived shortly thereafter that day, along with two transformers and more than 1,000 feet of cable. Each self-contained power module includes a diesel-fueled engine coupled to a generator, an 1,100-gallon fuel tank, generator controls, relays and floor-standing switch gear. Technicians from the local gen set supplier and an electrical contractor also arrived on-site early in the day to connect the equipment, which was placed at the base of the utility pole. For safety reasons, a four-foot-high temporary fence was built around the temporary power site.
The three power modules were then paralleled to provide the needed 5.25 MW, and the two temporary transformers were installed to step up power from 480V to 13,800V and match the 2,600-amp requirement of the utility transmission lines. The transformers, connected to the gen sets via four sections of cable and fastened with bus bars, were then attached to the utility pole's stress cones with 50 feet of high-voltage copper wiring. From there, the transmission lines fed power directly to the plant's substations, where transformers stepped the power down to 480V before the load was transferred to the plant's electrical buses for final distribution. All told, nearly 600 feet of cable connected the rental equipment to the plant substations.
Production Resumes, Downtime Minimized
After testing the temporary power configuration, the gen sets officially began powering the plant at midnight on Wednesday, March 1, only six hours after arriving on site. With power supplied by the rental equipment, maintenance crews restocked materials and started plant equipment during the night of Thursday, March 2. Normal production resumed at 6 a.m. on Thursday and the plant ran at full capacity on temporary power during the first nine-hour shift back on the job.
Temporary power continued to power the facility without so much as a flicker until 11 a.m. on Friday, when the plant was notified that utility backup power had been restored. Throughout the operating period, the gen set supplier's technicians remained on-call to handle any unexpected emergencies, periodically visiting the plant to monitor equipment. Technicians disconnected the high-voltage wiring coming from the transformer to the utility pole's stress cones during the plant's lunch hour on Friday. Power was then transferred back to the utility grid and production resumed.
Because the power situation was still unstable, two of the three power modules ran in parallel with the utility for another week to pick up the plant load if needed. The third power module was delivered to a local stamping plant that was still without utility power because a circuit breaker was under repair. The rental unit powered the plant, which is a supplier to the Volvo operation, until 6 a.m. on Saturday, March 4, when utility power there was restored.
Power Contingency Plans Solidified
Every cloud has a silver lining. In this case, the unexpected utility outage taught Volvo several important lessons. The first is to have airtight contingency plans for maintaining essential operations during unexpected outages. Our plan now calls for the digital telephone lines and LAN-based computers to be shut down immediately during outages to preserve power from the permanent, 36 kW generator for the plant's computer-based inventory control system. This will allow the system to operate until gen sets are installed. Meanwhile, the plant will use lower-powered analog phones to maintain communications. The plant will also run portable welders, equipped with 10 kilovolt (kVA) gas engines, to power the radio scanner charging system used by the stepped-up security force and to run emergency lights.
Another lesson learned is to have a temporary power plan in place. As part of that plan, the response team first determines the plant's aggregate electrical load needed to keep the plant operating. Our next step is to decide where the gen sets should be located at the plant so they can be rapidly installed and not interfere with plant operations. After that, power distribution requirements are considered, including determining how the equipment should best be hooked up and both the voltage and ampacity needed to connect the equipment to the plant's electrical system. We then can contact a local temporary power supplier who can provide the equipment and technical support to get power running in minimal time.
To help hasten the installation of the rental equipment, the cable, lug ends and copper wiring needed to connect the rental gen sets and transformers have been boxed and placed in the plant's storage areas.
In the event of another power outage, the pre-cut cable can be connected to rental equipment in less than two hours, instead of taking upwards of six hours.
Looking back, the unexpected loss of power could have been devastating to the Orrville plant. However, by acting quickly, following the proper steps, securing utility-grade temporary power equipment from a qualified supplier and good-old fashioned teamwork, a potentially huge problem was minimized. In all, the plant was without power for less then 24 hours, while other businesses in the city had to endure life without electricity for days. Additionally, Volvo GM Heavy Truck Corporation in Orrville is more than ready to handle the next unexpected outage, which as we learned, can strike when least expected.
Hank Jeanneret is the Facility and Maintenance Manager with Volvo GM Heavy Truck Corporation, Orrville Assembly Plant
This article adapted from Vol. 9#2.