Data Is Saved Online, But Rarely Used
Unprecedented data growth is threatening disaster recovery plans. Companies in all industries rely on large complex relational databases to support mission-critical applications. It is not uncommon for customer-facing and transaction-intensive applications across industries (insurance, healthcare, telecommunications, financial services, transportation, manufacturing, utilities, and others) to become data magnets. Companies in these industries must provide access to customer data and ensure that customer service levels are met. Companies must also retain data for specified periods of time to comply with business policies and government regulations.
One of the common characteristics among all of these industries is that the large volumes of data remaining online in production databases are rarely used. CRM, ERP, financial and other high-transaction applications are already overloaded with data that is growing exponentially, with many relational databases adding terabytes of data annually. Storing years of rarely-accessed, nonessential data online expands backup and database maintenance windows and increases the amount of data that must be restored just to get vital data back online, slowing the recovery process by hours or even days.
There is a direct correlation between the size of the database and the speed of recovery. The bigger the database, the longer it will take to recover vital data and restore service. Without an easy, safe mechanism for separating critical and non-critical data, it will become increasingly difficult to meet disaster recovery service level agreements (SLAs). Could your company afford a long delay at a critical time when your customers are depending on your systems to be up and running?
Traditional Approaches To Disaster Recovery
The traditional approaches to disaster recovery focus on speed. Common techniques include implementing more efficient backup and recovery tools, increasing network bandwidth and throughput, and adding faster hardware. As databases continue to grow, faster and faster systems are needed to maintain SLAs. Although these strategies are essential to ensure faster recovery and to restore service as quickly as possible, essentially they do not address the problem of overloaded databases, but rather focus on the effects, providing only a tactical approach to reducing disaster recovery time.
A key distinction is that traditional approaches do not separate business-critical data from nonessential data. Instead, both essential and nonessential data is copied from a production database to a “recovery” storage medium. As a result, the recovery process will take longer. For example, even with the most sophisticated disaster recovery plan, an insurance company will still have to recover 10 years of closed claims to get open claims back online. Clearly, a new approach is needed.
Advances In Database Archiving To Speed Disaster Recovery
As mission-critical databases continue to grow, the amount of data a company needs to recover can be daunting, and it is essential to develop a plan for recovering mission-critical data first.
Active archiving was developed because organizations required a new approach that addresses the root cause of the problem – expansive database growth. This requirement is compounded by the inherent complexities of managing data in a relational database, where it is normalized across hundreds of tables, interconnected by hundreds of relationships. Data from complex relational databases must not only be archived, but it must be archived in such a manner that the business context is retained, ensuring the value of the data.
The key strategy for disaster recovery is to reduce time to recovery by separating inactive from active data, while maintaining operating databases at a manageable size and ensuring that the data remains easily accessible.
Active archiving allows companies to remove rarely-accessed data from production databases and manage archived data efficiently, while providing easy access to the archived data on demand. Archived data is saved to an archive file – referentially intact and complete – and these files can be easily saved to a more convenient and cost-effective medium. Once archived, the data can be safely removed from production databases, improving the performance and availability of critical systems immediately. If needed, the archived data can be easily accessed, researched and selectively restored.
By separating essential data from nonessential data, many companies can safely reduce the size of overloaded databases by up to 50 percent or more during the initial archive. If four hours are needed to restore a 500 GB database, then active archiving can reduce this time to half by reducing the size of the production database to about 250 GB or less. Because databases are streamlined, there is much less data to recover. Most importantly, active archiving provides both a “best practice” strategy and proven technology that can significantly improve a company’s ability to get critical business applications up and running as quickly as possible in the event of a disaster.
Ongoing active archiving (daily, weekly or monthly) enables companies to maintain mission-critical databases at a size that fits comfortably within the desired recovery window, while still meeting all business needs. Routine backup and database maintenance processes take much less time. Mission-critical data is available at peak performance. By effectively managing database growth, companies can ensure their ability to meet disaster recovery SLAs.
In the event of a disaster, the recovery process can be staged by recovering mission-critical data first and nonessential data later, if necessary (see graphic). The staged approach significantly reduces the amount of time and resources needed to rebuild an alternate database, resulting in a shorter time to recovery and operation of critical systems.
Once vital systems are recovered, a phased recovery plan can be used to restore archived data, if needed, based on the company’s business priorities.
Because active archiving supports all of the leading relational databases in both distributed and mainframe environments, the technology can be applied to all applications (CRM, ERP, purchased and in-house) to enhance disaster recovery plans across the enterprise, using a common strategy that is tuned for each application.
Implementing Active Archiving For Disaster Recovery
Here is how one Fortune 500 corporation implemented active archiving to improve disaster recovery for their enterprise CRM solution. This mission-critical application allows customer service agents to log, track and manage customer data stored as application objects, called incidents. Several years of closed incidents were stored in the production database along with current, open incidents. In the event of unplanned downtime, all of the closed incidents had to be restored just to access open incidents.
The cost of downtime associated with this 24x7 worldwide CRM application was estimated in the millions of dollars per hour. Management evaluated the company’s disaster recovery plan and found that it would take several hours to fully recover the application. A corporate decision mandated that the company could not afford to risk unplanned downtime of more than two hours. To avoid this risk, IT management determined that the only method for achieving this business objective was to think outside the box and look for new ways to solve the problem. The result was to use active archiving to reduce the size of the database to be recovered and greatly reduce the time to recovery, providing the only option for meeting their disaster recovery SLA.
The company anticipated the CRM database was growing at a rate of 40 GB per month and that once the database size surpassed 200 to 250 GB, the disaster recovery window would exceed the two-hour SLA. They recognized that active archiving would enable them to meet this SLA by keeping the CRM database within the 200 to 250 GB window.
During the initial archive process, active archiving safely removed closed incidents older than 30 days, significantly reducing the size of the CRM database. Active archiving now provides the capability to easily remove between 25,000 and 50,000 closed incidents per day to maintain the database size within the 200 to 250 GB range. Archived data is easily accessible and can be restored to production even if the data model has changed. The company can rest assured that their financial risks associated with unplanned downtime will be contained within the two-hour SLA. Active archiving provided the strategy and technology to effectively solve this problem for the long-term.
Norwich Union Insurance Limited, a leading European insurance company, also chose active archiving. Business expansion and corporate acquisitions created large increases in database growth for the company’s mission-critical insurance underwriting application. Continued database growth would not only degrade response time, but would also extend the batch processing window and adversely affect system disaster recovery times, which are covered under SLAs. Norwich Union realized the need for an effective strategic solution to address these issues. Active archiving was selected to ensure SLAs across the enterprise.
Enhancing Backup Processing
When discussing active archiving, it is important to note that this technology is designed to complement routine backup procedures but not replace them. Today companies use a variety of technologies to backup data including electronic vaulting and journaling, database mirroring and shadowing, and hot standby or load-balanced solutions. Active archiving is the ideal complement to any of these solutions.
Most importantly, active archiving ensures data quality and referential integrity. Active archiving understands and remembers data relationships with 100 percent accuracy by saving not only the data, but also the metadata describing the tables, columns and relationships used to create the archive file. Users can still take advantage of real-time access to archived data using comprehensive search, browse and reporting capabilities without restoring a single row to the production database.
If the data must be restored, users no longer need to restore an entire archive to restore a small amount of data. Data may be restored selectively to the production database using criteria that may be different from the criteria used to create the archive. Active archiving also ensures that data can be restored to a new or existing database, even if the data model has changed over time. The benefit of selective restoration is magnified in a disaster recovery situation when an efficient method to retrieve only needed data saves precious time and resources.
By keeping production databases streamlined, active archiving improves application availability. The smaller the database, the faster it can load, unload, search, reorganize, index and optimize. In addition to disaster recovery situations, this holds true for planned downtime (upgrades and maintenance) and unplanned downtime (hardware or software failures, database corruption or power loss). Because downtime can cost companies millions of dollars, active archiving delivers a greater ROI for applications that rely on complex relational databases.
Summary
A disaster recovery plan for mission-critical applications is essential because unplanned downtime can result in millions of dollars in losses. Active archiving provides a “best practice” strategy and proven technology to safely separate essential and nonessential data. This approach significantly improves recovery time by focusing on restoring priority operational data for critical systems immediately. Active archiving enables companies to recover operational data first and then recover archived data, if needed, after the critical systems are operational. An effective active archiving strategy improves availability in the event of any type of downtime – planned or unplanned – and can significantly reduce recovery time in the event of a disaster.
Jim Lee is vice president of product marketing for Princeton Softech (www.princetonsoftech.com), the pioneer and leader in the active archiving market. Lee has extensive experience in product management, application development and consulting.
