The State Of Disaster Recovery Preparedness
- Published on January 6, 2011
- Written by RACHEL DINES
Forrester Research and the Disaster Recovery Journal have partnered to field a number of market studies in business continuity and disaster recovery (DR) in order to gather data for company comparison and benchmarking and to guide research and publication of best practices and recommendations for the industry. This is the fourth annual joint survey study, and it's focused on gathering a baseline of company DR preparedness. This study repeated many of the questions that we asked in 2007, to determine what has changed in DR in the past three years. Specifically, this study was designed to determine:
- How much companies are spending on DR.
- Company practices regarding DR planning, DR plan maintenance, and DR testing.
- The percentages of companies that have alternate recovery sites, the number of sites, and the distance between sites.
- Current recovery objectives and technology selection.
- Company confidence in DR preparations and preparedness.
- The most common causes of disaster declarations and downtime and the cost of downtime.
- Market drivers fueling continued improvement in DR preparedness.
In the fall of 2010, Forrester Research and the Disaster Recovery Journal (DRJ) conducted on online survey of 200 DRJ members. In this survey:
- Forty-three percent of respondents were from companies that had 0 to 999 employees; 25 percent had 1,000 to 4,990 employees; 19 percent had 5,000 to 19,999 employees; and 14 percent had 20,000 or more employees.
- All respondents were decision-makers or influencers in regard to planning and purchasing technology and services related to disaster recovery.
- Respondents were from a variety of industries.
This survey used a self-selected group of respondents (DRJ members) and is therefore not random. These respondents are more sophisticated than the average. They read and participate in business continuity (BC) and disaster recovery publications, online discussions, etc. They have above-average knowledge of best practices and technology in BC/DR. While nonrandom, the survey is still a valuable tool in understanding where advanced users are today and where the industry is headed.
Disaster Recovery Budgets Took A Downturn With The Economy
It isn't surprising that when the global economy began to decline in 2008, DR budgets declined sharply as well. Overall, DR spending has declined since we last fielded this survey in 2007. According to our survey, 53 percent of respondents spend less than $500,000 annually on disaster recovery (see Figure 1). This is down from the results of 2007, in which only 45 percent of respondents spent less than $500,000 on DR annually. However, according to Forrester's Global IT Budgets, Priorities, And Emerging Technology Tracking Survey, Q2 2010, 32 percent of enterprises and 36 percent of SMBs plan to increase spending on BC/DR by at least 5 percent.
Disaster Recovery Planning, Maintenance, And Testing Has Remained Flat Or Regressed
Perhaps one of the biggest challenges to effective DR preparedness is planning, maintenance, and testing. Through client inquiries and advisories, Forrester finds that many companies don't have formal plans in place, and if they do, these plans aren't integrated into regular configuration management and change management processes, and they aren't regularly tested. While the number of companies that had DR plans in place stayed flat at 79 percent of respondents between 2007 and 2010, survey results indicated that companies update the plans less frequently (see Figure 2-1). In 2007, 58 percent of companies were updating their plans at least twice a year, but in 2010 that number shrank to 42 percent (see Figure 2-2). The best practice in plan maintenance is to ensure that DR plans are continuously updated as part of configuration management and change management.
However, despite companies updating their plans less frequently, the frequency of plan testing has remained flat, at almost a quarter of companies testing twice a year and around half of companies testing once a year (see Figure 2-3). Forrester recommends that companies conduct at least two full tests per year, followed by several component tests throughout the year.
Companies Are Consolidating Secondary Sites, Continue To Prefer Dedicated Infrastructure
Over the past three years, companies have been consolidating their recovery sites. While approximately the same proportion of surveyed companies have secondary sites – around 85-90 percent – the number of secondary sites that companies have has decreased. In 2007, 33 percent of organizations had more than one secondary recovery site, but in 2010 that number dropped to 27 percent (see Figure 3-1). This reduction in DR sites is most likely directly attributable to the declining or flat budgets that have plagued disaster recovery for the past few years. This consolidation is not necessarily a sign of poor planning or lack of preparation — many organizations' recovery time and recovery points don't justify having a three-site configuration, as long as the sites are separated geographically.
Companies clearly continue to prefer dedicated infrastructure. According to our survey, 46 percent of organizational are using an internal site, 17 percent use a colocation site, 9 percent use a dedicated fixed-site provider, and 7 percent use dedicated managed hosting (see Figure 3-2). With dedicated infrastructure, companies can achieve a much better time-to-recovery and improve their recovery point with the use of replication.
There is no rule of thumb when it comes to the appropriate distance between your data center and your recovery site. You have to achieve enough distance between sites to escape the same set of threat events while balancing recovery requirements, technology limitations, and cost. We found that between 2007 and 2010, survey respondents reported more moderate distances between primary and secondary data centers. In 2007, 20 percent of respondents reported that the distance between their primary data center and furthest backup data center was greater than 1,000 miles. While in 2010, only 13 percent claimed this distance (see Figure 3-3).
More And More Applications Are Considered Critical
When asked to classify the percentage of applications that fell into mission-critical, business-critical, and non-critical categories, companies continue to favor mission- and business-critical classifications. On average, respondents classified approximately 36 percent of applications and data as mission-critical, 36 percent as business-critical, and 29 percent as non-critical. The survey shows that:
- Tape is still the dominant recovery technology. For all applications and data sets, regardless of criticality, tape still remains an important part of disaster recovery preparedness. This means that while the use of replication is increasing, companies will still perform traditional backups as a precaution and to ensure that they have a history of recovery points.
- Companies are expanding the use of replication. Replication is no longer used only for mission-critical applications. The survey indicates that companies use synchronous and asynchronous replication to protect business-critical applications (see Figure 4).
Recovery Times Are Longer, But Data Loss Remains The Same
Despite technologies such as virtualization making DR less expensive and easier to implement, our survey found that reported recovery times from disasters or other major business disruptions have lengthened (see Figure 5-1). In 2007, 30 percent of survey respondents reported that they were able to recover with no downtime, while in 2010 that number shrank to 13 percent. On average, however, recovery times remained fairly flat, at 17 hours in 2007 and 18.5 hours in 2010.
The amount of data lost during disasters and other major outages, however, remained fairly flat between 2007 and 2010. In 2007, 62 percent of survey respondents reported no data loss after their most recent disaster or disruption, while in 2010, this was reported by 58 percent of respondents (see Figure 5-2). The average amount of data lost actually went down slightly, from 6.3 hours in 2007 to 4.8 hours in 2010.
Most Declared Disasters And Disruptions Are Preventable
According to our survey, approximately 76 percent of companies have never officially "declared disaster" in the past five years. Companies should not take comfort in this statistic. It still means that almost a quarter of companies are likely to declare disaster and failover to their alternate site in a five-year time period, and it doesn't take into account the events that disrupt operations but don't affect the entire data center. Although only 24 percent of survey respondents have declared a disaster and failed over to an alternate site in the past five years, an additional 40 percent of respondents do admit to having some sort of major disruption to their business operations. The survey also found that:
- The most common cause of declared disaster is power failures. Forty-four percent of respondents indicated that a power failure was the cause of their most significant disaster declaration or major business disruption, followed by IT hardware failures, and network failures (see Figure 6).
- Forty-seven percent of companies claim to have calculated their cost of downtime. Calculating the hourly cost of downtime is a task that many companies struggle with. Determining productivity losses, lost sales opportunities, and compliance penalties is easier to do, while calculating impact on customer retention, reputation, productivity, and morale is much more difficult. Even though 47 percent of survey respondents claimed their companies have calculated the cost of downtime, only 15 percent actually knew what that figure was. The average reported cost of downtime per hour was almost $145,000.
- Eighty-six percent of companies don't know the cost of their most recent declared disaster or disruption. So IT operations has improved planning, maintenance, testing, and actual response but still can't actually measure the cost of a declared disaster. For the 14 percent of companies that do know their costs, the average total cost of a declared disaster is approximately $1.4 million.
Companies Overconfident In Their DR Preparedness, But Still Strive For Improvement
Despite evidence to the contrary, companies are more confident in their DR capabilities today than they were in 2007. In 2007, 15 percent of survey respondents stated they felt very prepared for a site failure or disaster event, but in 2010 that number climbed to 23 percent (see Figure 7-1). Considering that investment in DR has declined from a monetary and time standpoint and that recovery time and recovery points have grown, this assurance appears to be unrealistic. Despite this overconfidence, there continues to be a push to improve recovery capabilities even further, with 45 percent of survey respondents stating that improving overall DR preparedness and capabilities is very critical, and 43 percent and 39 percent stating improving RTOs and RPOs respectively is very critical (see Figure 7-2). However, when reviewing these numbers, it is important to remember that the survey respondents are a self-selected group of respondents (DRJ members) and is therefore not random. According to Forrester's Technology Forrsights For Hardware, Q3 2010 survey, only 15 percent of respondents reported upgrading BC/DR capabilities as a critical priority and another 45 percent reported it to be a high priority.
When respondents were asked to identify what was driving their need to improve disaster recovery preparedness, fiduciary responsibility was ranked the highest, followed by regulatory or legal drivers, and the need stay online and competitive (see Figure 8). The survey results indicate that DR preparedness is a priority, but that in recent years investment in it has declined due to outside factors. Moving forward, as the global economy recovers, investments of both time and money in DR preparedness will improve, as it clearly continues to be a priority with strategic senior executives, stakeholders, partners, and customers.
Rachel Dines is an analyst at Forrester Research serving infrastructure and operations professionals. Her research focus is on IT continuity and disaster recovery services and technologies, next-generation high availability and backup, and data center strategies. Additionally, Dines conducts research on infrastructure and operations metrics and organizational structures.