DRJ's Spring 2019

Conference & Exhibit

Attend The #1 BC/DR Event!

Spring Journal

Volume 32, Issue 1

Full Contents Now Available!

logo

Business Continuity and Disaster Recovery Planning Templates

The fully editable and customizable templates provided below offer sample content and procedures to help jumpstart your business continuity and disaster recovery planning effort. Remember, templates are just a starting point and should be customized as necessary to fit the unique needs of your organization and program.

catalyst

Your Download Includes:

When tragedy hits, we want the world to stop.

But even the most senseless tragedy does not stop the world. Not for you. Not for me. Not for a bereaved community in Connecticut, where schoolchildren were slaughtered in their classrooms.

No, it doesn’t stop. I know this as a fact.

When a police officer knocked on my door one midnight to inform me of the death of my young daughter, my world stopped. But the next day, one of my dear friends had a medical crisis, and I needed to step up to support her family. And the next week, my favorite restaurant across from my condo had a fire, and I needed to run across the street to help friends out of harm’s way.

Crisis didn’t stop in the middle of my wrenching grief. And, though time has passed, my grief hasn’t ended. It has changed. Somewhere in my anguish, I knew that I had two choices: 1) to pack up a U-Haul and move my furniture into the tomb of my pain, or 2) to make it matter. To make it matter, I have to stand up to it every day. I have to re-evaluate my faith every day.  I have to stand up and suit up and show up every day.  I visit the grief constantly, but I do not reside in it. I can’t. It would end me.  And so I make it a proactive choice to do my best to teach this to others in the face of their most heinous moments because, any moment now, a next disaster will arrive somewhere. I hope it won’t be yours.  

To make a heinous disaster meaningful is dreadfully hard work forever because it doesn’t fix the grief.  What it does is transmute the energy of sorrow into something more valuable than the pain -- hope.  But it isn’t easy to get there. It is a marathon, not a sprint.

When the next tragedy happens, we re-grieve all our previous losses. We join in the collective of our own sorrows and those of others. And there will be, unfortunately, another event just around the corner. It is sadly inevitable. So I encourage you to prepare your hearts.  Prepare them not by closing them, but by opening them more.  It isn’t easy work. But the only other option is to crawl into the tomb with your losses, and then there is no hope left.

I wish I didn’t belong to the “special” club of parents who have lost children – that club in which 20 families in Connecticut now find themselves unwilling members. And I can’t forget that my Mom died three weeks after my daughter -- not a good year for me! Not an easy club to belong to. But I do. I remember sitting across the table from parents who had lost a loved one in the World Trade Center attacks in 2001, people who cried out to me, “You don’t know what I’m feeling!”  I had to say, “Yes, I do. And would you like to know what I have done to survive?” Some did want to know.  Others were already ordering up their U-Hauls to move in with the pain.  I don’t blame them.  That tomb looks pretty inviting sometimes.  But if we go there, the “bad-guys” win. And I refuse!

Suit up and show up. That’s all we can do for each other.  It won’t feel like enough. My heart is re-broken after the school massacre last Friday. And nothing I can say will help, except to repeat: I know.  I know.  

And I will stand up again today to try to make the day worthy of my survival.

josh-stevensAccording to numerous studies, small businesses are much more likely to fail within a few years after a disruption than larger businesses. One reason this occurs is that small businesses lack the money to prepare for risks. Large businesses have the funds to pay for businesses preparedness, such as business continuity, and are more likely to survive disruptions. However, there are some simple steps that a small business can take to manage such risks.

First, the small business owner needs to determine what the risks of the business are. This can be performed by identifying the natural hazards that occur in the businesses’ geographic area, such as hurricanes in Florida. Also, identify the risks that could occur at the business’ site or facility, such as fires and power outages. Once the risks that threaten a business are understood the risk management process can begin. Depending on the cost of the business, you can determine how much money you are willing to spend on managing the risks. Some risks may cause such little damage that it would be easier and cheaper for the business to recover from the risk once it occurs, while other risks may be so costly that insurance might be cost efficient for the business.

Next, let’s go over a few of the common risks that all businesses must contend with, which are the loss of a key business supplier or vendor and critical documents. Here are some basic steps to manage supply chain disruption:

  • Gather the contact name, phone number, and email address of the key contact for your suppliers and vendors in case a disruption does occur
  • Contact your key vendors and suppliers to find out what services they will provide during emergencies and how they will reimburse you if they cannot meet their contractual obligations due to a business disruption
  • Identifying and contracting with alternative suppliers and vendors as backup if your current suppliers cannot service you

Lastly, critical documents are vital to a business operations and legal requirements. These documents could be insurance forms, business licenses, or service contracts. A few steps to protecting these critical documents are:

  • Placing these documents in a fire and/or water resistant safe; this can also protect the documents from airborne particles which deteriorate the documents over time
  • Making copies of the documents and keeping the originals at your business or at an alternative site, such as your home or band safe deposit box
  • Store the documents off-site at a critical document storage facility which specializes in critical document protection
  • Locate a critical document restoration company and gather their contact information in case your critical business documents are damaged; a document restoration company can salvage documents that have been damaged by fire, water, mold, etc.
  • Purchase a back-up hard drive to back-up your critical electronic files on a scheduled basis

In conclusion, as the aftermath of Hurricane Katrina showed, individuals and businesses cannot only rely upon FEMA and other government agencies to look out for their best interests in regards to disruptions and disasters. One step we can take is to prepare for and manage risks ourselves.

 

Josh Stevens lives in Miami, FL and recently started Stevens Consulting there.  He has 6 years of experience in business continuity and risk management.  He is completing a Graduate Certificate in Business Continuity from Boston University.  You can contact him at (815) 999-2232 or at This email address is being protected from spambots. You need JavaScript enabled to view it.

  • Timing: have a plan, but don’t wait for it to be ‘perfect’ before beginning an exercise strategy

    • Successful drills and exercises are built from functional crisis, emergency and business continuity management plans. What does that mean? An effective exercise is the result of plans that are usable, in-progress, and somewhat up to date. So if the last person to touch the plan retired five years ago, it’s best to put a hold on any drills or exercises. Go back to the basics, re-draft the plan, and then think of an exercise strategy.

  • Clear objectives based on the business continuity plan

    • No plan is perfect, but it should at least reflect the critical elements of your company and industry. It needs to account for those areas susceptible to crisis in order to mitigate vulnerability. Also, objectives need to be clear when deciding on exercises and drills: does our company have critical assets in a particular location? How does the plan mitigate risks there? Maybe it is time to test the supply chain to see whether employees and the plan can handle a blow to key assets.

  • Find the right venue and partner to fill weaknesses in expertise

    • Sometimes it’s hard to know when in-house crisis management planning needs to make room for outside experts. In order to get a clear action plan for improvement, it’s best to have a dedicated resource to run company exercises. Yet some crisis management consultants will perform no better than in-house talent. An expert , however, will understand what to ask and how to build scenarios that test your enterprise the right way. And if you have clear objectives, you can more quickly weed out internal and external ‘non-experts’ from your team.

  • Scenario planning to actually challenge participants

    • The term ‘powerpointless’ didn’t come from nothing. Every industry has training sessions built from static slides—we’ve all sat through them. Beyond one or two catch phrases, most employees walk away little actionable improvement. While good scenario planning builds upon clear test objectives, excellent scenario planning takes objectives and creates believable, timely simulations to bring out the worst and best in teams and plans. We’ve seen firsthand how companies run drills and completely miss a key scenario element—such as the growing influence of social media—or botch the exercise by excluding individuals.

  • Evaluation standards for transparent results

    • Clear objectives create clearly measurable results. It’s that simple. Benchmarks provide a clear-cut way to analyze critical components of the plan and its participants, both during and after the exercise. A thorough analysis will not just look at the scorecard of performance versus metrics; it will also bring in outside expertise to suggest best practices and next steps for improvements.

There’s a common misperception in the crisis management & business continuity world about drills and exercises. Many businesses look at exercises as a sprint—holding one-day sessions and checking boxes on yearly prevention planning to-do lists.  But corporations don’t operate in sprints.  They don’t start up and die off quickly, even in today’s economy. Enterprises build themselves to run for the long haul: they are marathoners.  Like a marathon athlete, they need to plan far in advance, practice strategic training, and set milestones.  Of course, training for a race is different than training for a crisis, but the point to consider is preparedness. As such, it’s time to challenge the “sprinter” approach to crisis management and begin thinking about strategic exercise programs that are based upon a long-term, holistic perspective for corporations.

Because industries face uncertainty in today’s economy, business continuity professionals—now more than ever—argue for creative and unique approaches to planning for resiliency. After managing hundreds of corporate war games (and earning the buy-in of senior leadership), we’ve developed five pointers for strategic exercise programs.

1. It’s not enough to simply exercise. It’s a matter of what is exercised.

The value derived from testing a company’s emergency and business continuity plan is tied to the clarity and quality of the testing. In other words: garbage in is garbage out. A mere checklist of exercise “to-do’s” will probably not elevate preparedness, nor highlight areas of improvement. As William Gouveia shared in the DRJ Summer 2012 edition, organizations can have plans without true resiliency because their processes are unexamined. We understand this is difficult to do for those close to the plan. However, it is imperative to develop a critical and objective eye. Step back from the plan, and review it.

Be prepared, though, to accept that the plan is imperfect. Don’t wait to cross every t before initiating your strategic exercise plan. Also, create a list of clear objectives by considering the value that should be derived from a singular exercise, as well as from the overall exercise program.

Once there is a clearly defined list of objectives, it’s time to be honest: can these exercises, drills, or scenarios be created in-house? Can they provide results to meet the objectives? Knowing strengths and weaknesses is essential to success. Seek outside consultants if necessary; they can effectively lobby for an exercise plan and even help create a realistic set of scenarios. Of course, there’s the old joke that consultants “take your watches, tell you what time it is, and then charge you for it.” In no way should that be the case! Do the necessary research to find the experts who will work in partnership with you to reach your goals.

Whether outside help is brought in or the work is performed from within, scenario development should be the next important consideration. Besides the specific objectives, a realistic set of scenarios must take into account the company’s culture, region, organization, environment, and team. Another necessary key to creating an effective exercise is the format of the exercise, which in turn impacts likely scenario choices.

2. The type of exercise impacts what participants learn.

Determining the right exercise format is a key part of effective exercise planning. There are three general exercise categories: tabletop, functional, and full. The predetermined objectives, geographic location of participants and observers, and the demands of daily business all contribute to the format choice. Below, we developed an Exercise Format Grid to gauge exercises based on enterprise needs.

Exercise-Grid1

Figure 1: Exercise Format Grid

3. Choose exercise participants wisely.

This requires more than picking people with titles—it means looking across the entire enterprise. Systems theory, born out of Industrial and Organizational (I/O) Psychology, says the total health of an organization gets impacted when even one small area is weak. Thinking about how an enterprise functions means considering the whole rather than just the parts. Where are the potentially costly gaps in capabilities? Is it due to certain employees, or the plan? Both? Testing employees across divisions and studying the aggregate result will reveal the true health of a company.

It’s important to note here that senior leadership provides critical influence when it comes to employee participation. The exercise program will more quickly gain acceptance from the employee base if senior leadership communicates enthusiasm and support for the program. Researchers from I/O Psychology, business, and technology fields often note that for enterprise programs or cultural shifts to be effective, the drive must come from organizational leadership. Thus, senior management participation is ideal. If they cannot attend due to schedule demands, leaders should—at minimum—indicate support for the exercise program.

Recognized exercise expert Kathleen Lucy, noted in her recent DRJ article the importance of “bench depth” in an exercise, which comes from having as many employees as possible participate. Also, to achieve an overall representation of your actual company, attention must be paid to the mix of employees involved in the exercise. This can be accomplished many ways: divide and conquer department by department, or set up exercises by partitioning teams from across the enterprise. Critical areas that should always be involved in exercise programs include:

  1. Operations

  2. Communication

  3. Management (Mid to Senior)

  4. Critical Vendors

  5. Supply Chain

  6. Technology

Put simply, consider the entire organization when setting up a strategic exercise plan. Think horizontal as well as vertical.

4. Practice with the right frequency for optimal results.

The question of when to have an exercise depends on objectives, of course. Quite frankly, a one-time exercise will accomplish little. However, this doesn’t mean companies should do a blast of training in one fell swoop, either. It’s about setting an exercise program strategy—with the right mix of exercises and drills—at the right intervals. To go back to our initial analogy, this is marathon training. Elite marathon athletes run a variety of distances and speeds; they also integrate cross training with weight lifting, cycling, swimming, etc.

Of course, exercise programs are an investment. And like any good investment, some of the expense must be paid up front before any value can be derived. We’re conscious that exercise programs can seem costly to an enterprise. However, we’ve all seen the price companies pay when a crisis finds them unprepared. Exercise programs should be the right balance of capital and time. With clearly defined objectives at the start, companies can more comfortably measure the investment against the results. This can’t be easily quantified with a one-off training session, or a predictable annual exercise.

After considering the investment and the objectives, it becomes easier to determine when certain exercises should take place and with what frequency. Let’s pretend Company X needs to focus on four key areas: supply chain, communication, senior leadership, and operations.  To do this effectively, it will involve 150 employees in three different locations. The geographic spread and specific objectives of Company X guide the plan for three tabletop exercises, one functional exercise, and one full-scale exercise. This mix gives economic and geographic flexibility while accomplishing long-term learning objectives. Though it is hypothetical, this scenario shows it is possible to balance time and budget costs in planning.

5. Where you practice impacts your experience and results.

When planning an exercise program, environment plays an important part because location impacts the effectiveness for attendees. It can also affect evaluation and future recommendations. For example, if an international company wants to run a realistic scenario for 100 of its employees, should they all gather in the large auditorium at company headquarters? A virtual solution would allow teams to be geographically spread out, just as they would be during a real-life event. More and more, global corporations are incorporating virtual solutions as part of their exercise program strategy. A real-life environment is key—crises don’t occur on PowerPoint slides! Note the Exercise Format Grid again, with Virtual Tabletop highlighted.

Exercise-Grid2

Figure 2: Virtual Tabletop solutions provide cost effective exercise value

To summarize, organizational resiliency depends on not only a plan, but on a strategic process, which tests, measures, and evaluates preparedness across the entire enterprise. Just as a marathoner strategically trains to prepare for a race, an enterprise needs to implement training, drills, and exercises as part of its business continuity strategy. With the proper goals, scenarios, attendees, locations, and frequency, an enterprise can be prepared for what comes. That’s the value in strategic exercise programs.

RobBurtonRob Burton has been leading groups and teams since his early days in the United Kingdom Special Forces; he is currently a co-founder and director of risk management at BWP Global. Burton has been featured on FOX News and has authored several articles for industry publications including the Disaster Recovery Journal. He can be reached: This email address is being protected from spambots. You need JavaScript enabled to view it.


TomChiginskyTom Chiginsky is an internationally recognized expert in internet computing, content delivery and collaboration. He is a co-founder of BWP Global, where he also serves as chief technology officer. Tom has also participated in international exercises focused on turning global data into knowledge to interdict and reduce the threat of WMD. He can be reached: This email address is being protected from spambots. You need JavaScript enabled to view it.

Exponential data growth is nothing new to today’s large enterprise data centers. For the past several years, data has grown at unprecedented rates, pushing IT managers to find new, more efficient ways to move, manage, and protect their data. However, with the increased adoption of very large databases and the advent of Big Data technologies, this already extraordinary growth rate is pushing backup and disaster recovery systems to a critical point. We are beginning to see the effects of this trend in the dramatic increase in data center “sprawl”, increased difficulty in meeting backup windows and replication windows, and greater emphasis on driving data center cost-savings and efficiency. A recent vendor survey – the Enterprise Data Protection Index 2012 reveals the challenges and priorities for disaster recovery in large enterprises and big data environments.

According to the survey, data growth is unabated. Thirty-three percent of respondents reporting that their data was growing at 20-30 percent annually and an additional 20 percent reporting even higher annual growth rates. Respondents also reported a marked increase in data growth compared to last year. Nearly one-quarter of respondents reported a 25 percent higher growth rate compared to last year.

More options for DR

In the past few years, we are seeing disaster recovery strategies move from an almost universal use of physical tape libraries to a much more mixed use of technology. While making copies to physical tape and shipping them off-site is still in use at 18 percent of companies, more and more companies are also using disk-based backup and electronic replication for DR. Nearly half (47 percent) of respondents are replicating more than 50 percent of their data to remote location for DR protection.

With this move to disk-based disaster recovery, companies are also moving to active-active strategies for improved RTO/RPO. According to the survey, 21 percent have an active-active remote replication strategy in place and 41 percent have an active-passive replication strategy.

Solutions Lead to New Problems

In many cases, enterprises have tried to solve the challenges of disaster recovery by replacing physical tape with single-node disk-based backup appliances. These systems – which were designed for medium-sized companies -- solved some of the problems of physical tape. They sped up backup performance and improved reliability by eliminating tape drive failures, and reduced capacity requirements through inline deduplication. Unfortunately, large enterprises and big data environments have data volumes that these systems cannot handle efficiently. These systems force companies to buy a new, independent system every time they need more capacity or performance, resulting in costly data center sprawl. Inline, hash-based deduplication in these systems are also problematic for large data volumes as they slow backup and restore performance and typically provide poor capacity reduction in large database environments.

With data divided among multiple “siloes” of storage, the disaster recovery schema for many large enterprises quickly becomes complex, costly, and prone to human error. Fifty percent of survey respondents characterize their environments as having “moderate” or “severe” sprawl requiring them to routinely add data protection systems to scale performance or capacity. These systems also use a hash-based, inline deduplication technology that is quickly overwhelmed by large data volumes.

Scalable data backup and disaster protection solutions are more efficient and cost-effective for large, complex environments in an efficient centralized way. These systems allow IT managers to add capacity and performance as their needs grow – enabling companies to protect petabytes of data in a single system. They are also built to deduplicate massive volumes of files and database data (a problematic data type for hash-based deduplication) without slowing backup or replication performance.

Remote Office Disaster Protection

Enterprises still struggle to find efficient ways to protect data in remote offices and branch locations. DR strategies for these locations vary widely. At the high end of the DR spectrum are hub-spoke topologies where data in remote offices is backed up to a small disk-based virtual tape library and replicated to a centralized backup system in a central data center, thence to a remote data center for DR. At the low-end, there is no formal DR strategy for remote offices and data is left unprotected. In fact 15 percent of data in remote offices and 11 percent of data in main data centers are currently not backed up or protected. In addition, a full 17 percent of respondents are still either working without a disaster recovery strategy or are in the process of implementing one.

While many smaller organizations have made marked improvement in their disaster protection strategies, disaster recovery is still evolving in the large enterprise data centers. TAs data volumes continue to climb, these organizations will need to move to scalable, enterprise-class technologies that can meet their needs for efficient backup, restore, and replication without causing costly sprawl.

Peter Quirk is a director of product management at Sepaton. He has spent most of his career working for vendors in systems engineering, product marketing, product management and project management roles, with responsibilities in operating systems, databases, languages, hardware platforms, storage, and social media. In his spare time he likes to code and explore the world of Big Data and all things related to Hadoop.