|
Aug 28
2010
|
BC Stories – Virginia Computer OutagePosted by: Glen Bricker in DRJ Blogs on Aug 28, 2010   |
 |
One of the ongoing problems in business continuity is articulating the business case in a way that resonates with executives and others in our organizations and doesn't require a twenty slide PowerPoint presentation. In an attempt to help, I will periodically post items as I find them in the news or wherever they turn up. The idea is to give business continuity professionals ideas for informal conversations that highlight both the need for business continuity activities and some potential pitfalls of ignoring them.
On Wednesday, August 25th, a hardware failure in Virginia's main data center brought the services of multiple state agencies to a halt. The failure impacted twenty-six of eighty state agencies including the Department of Motor Vehicles and the Department of Social Services, two of the most visible, public facing agencies. Details can be found in an article from the Richmond Times Dispatch at http://www2.timesdispatch.com/news/2010/aug/27/8/vita27-ar-475161/. As of Monday morning, August 30th, three of those agencies still reported outages, including the inability of the DMV to issue driver's licenses.
The potential lessons to be learned from this situation are numerous; from the value and limitations of outsourcing in general to the cost/benefit of system consolidation to the need for plans that coordinate with outsource providers. However, the one area that stood out to me was the lack of a consistent and positive message from state officials.
Within the same article the head of the Virginia information Technologies Agency comments that the state is "...disappointed to a have failure, an outage of this magnitude" while also stating that an outage that impacted twenty-six agencies "...was of insufficient magnitude to warrant activation of backup systems at a duplicate computer center..."
Rather than positively explaining that contingency plans are in place and communicating capabilities, the official downplayed the loss of DMV and Social Services functions for multiple days. While I have no doubt that all involved were making every effort to solve the problems, the focus of effort seemed internal. Stakeholder facing statements seemed to come as an afterthought, if at all.
The thought for business continuity practitioners, especially in relation to technology contingency plans, is to look beyond the technology. Understand the stakeholders that could be impacted by an outage and ensure that a communication plan exists to address their issues. Especially in the case of public facing activities, develop a robust and well thought out communications plan.
While this article doesn't provide a prescription for solving your issues, hopefully it will give you an opening to start a conversation or spur you to develop your own stories - often the most effective way of getting a point across and influencing without preaching.
Communications with internal stakeholders should include facts about the situation, the response initiated by management, ways employees can report in to their managers, employee assistance programs offered, and how the event might affect operations over the next few days. Some tools to utilize when communicating with internal stakeholders include email, crisis hotlines, notification services, call trees, “ghost” websites and public media (radio/TV/internet). Redundant methods of communication should be defined, and employees should be aware of how they may be contacted (or how to obtain up-to-date information) in the event of a crisis.
Communications with external stakeholders should include facts about the situation, what the company is doing to resolve the incident and what each stakeholder can expect as a result of the incident (how it may affect them). Other appropriate information may include expected event duration and open issues that management continues to investigate (“unknowns”). Some methods to use when communicating with external stakeholders include email, press conferences, public media (radio/TV/internet), print statements and direct calls. Ultimately, the method of communication is often determined by the content of the message.
Undoubtedly the State of Virginia will have many valuable lessons learned from this situation.