Planning for the Recovery of Vendor Acquired Software

By Ray Dicasali

Today’s high level executives have placed greater importance on business recovery planning and testing to ensure their companies can continue to supply customer and business services, even under the most adverse conditions.
Recovery plans aimed at maintaining critical operations within all divisions are being created, dusted off, updated or recommitted in growing numbers.
When applied to computer operations, however, companies have found that generating plans to protect hardware and software investments is a challenging proposition.
The trend in the 1980s toward decentralization of business operations has generated a mix of decentralized computer operations, with solutions distributed and networked to numerous divisional levels.
With such connected yet independently-operated systems, the potential damage from a given disaster is growing exponentially.
Yet most companies, even though they relied on third party vendors to supply them with decentralized hardware and software systems, only focus inward to develop a recovery plan.
In doing so, they miss the untapped power of the vendors who developed their systems to help develop and execute sound, workable plans.
And they miss the key differences between in-house developed applications and vendor-acquired software that could seriously impact recoverability.

The Differences With Vendor-Acquired Software

Disaster recovery plans that treat vendor-acquired software like any other software system risk recoverability failure since these plans ignore the four key differences between vendor-acquired and in-house developed software.
First, the development standards utilized by each software vendor usually vary from internal approaches. This is evident not only in the code itself but also in documentation, training and other materials.
Second, the file and database architectures may represent different or even proprietary implementations that vary from other applications. Therefore, the embedded backup and recovery procedures will likely be different.
Third, the availability and timing of software modifications such as new releases, bulletins or program fixes can have an unpredictable effect on recoverability.
Fourth, vendors typically supply training and consulting support. This means internal groups may not be as familiar with the systems as they are with in-house developed applications.
Given these significant differences, the software user has two choices: increase internal resources to accommodate disaster planning for what is most likely a large staple of acquired software products; or call on vendors to provide this as a part of their ongoing service. Clearly, it is critical that vendors be involved in disaster recovery planning.

Gaining
Vendor
Commitment

The best time to gain vendor commitment is prior to the software purchase, since it will make recoverability a subject that the project team must focus on during the software selection process.
It will also ensure that the needs analysis and requirements definitions include the issue of recovery planning and support.
These requirements can be embedded in the selection process to guide software selection and screen potential vendors.
In analyzing responses to requests for information, it is critical to find vendors that treat recoverability seriously.
They must be willing to provide documentation and recovery procedures and should recommend their involvement in the planning and recovery processes, if necessary.
A good time to check a vendor’s track record in this area is during a reference visit. While few of the software references will actually have encountered a full-blown disaster recovery situation, a visit to their operations provides an idea of their preparedness and the vendor’s involvement in supporting that preparation.

The Planning Process
to Protect
Vendor-Acquired Software

There are a variety of methodologies that can be used to create a successful recovery plan. Most of these methodologies have a common set of steps that serve as a guide for companies and vendors to jointly achieve recovery objectives. The following, while not a complete list of steps involved in disaster recovery planning, represents those steps where vendor involvement is most beneficial.

Gaining
Management
Commitment

Most processes begin with the need to gain management and executive commitment, primarily because of the time and costs involved in establishing an effective plan.

The best time to gain vendor commitment is prior to the software purchase, since it will make recoverability a subject that the project team must focus on during the software selection process.

The commitment will be more easily obtained if executives are involved in the software acquisition process and if that process is focused on disaster recovery. If it is not, it will be necessary to re-establish awareness before proceeding with a plan.

Defining the Disaster Recovery
Process Team

The foundation of a disaster recovery planning process is built on the selection of the recovery process team.
It is critical that the team include those individuals in the technology and user community who have been instrumental in acquiring the vendor software with the most critical recovery requirements.
A simple way to do this would be to create a grid of all the applications in the company and a disaster impact of failures in each of the application areas.
Once complete, select those applications with the most critical recovery requirements and plot those against the sources of the software.
This will provide a good starting point for determining that subset of vendors that will be the most critical for support.
It will also, during the analysis of various disaster scenarios, help clearly assess which third party organizations will be involved in each scenario.
Most companies find common vendor software across multiple parts of the organization since the need for integrated applications makes companies acquire integrated families of software products.
If this is the case, it will certainly simplify efforts to coordinate planning with key vendors.

Preparing a
Readiness
Review

During the readiness review, organizations assess their current ability to initiate and carry-out recovery operations. This includes notifying the right individuals, assessing the damage and its impact, deciding what to do, issuing a recovery directive and monitoring recovery efforts.
A readiness review gives a preview of the planning work necessary to improve the level of required readiness. Key vendors can be very effective in readiness review preparation by providing input during risk assessment, current procedures review and critical system component identification
Vendor involvement at this stage has two benefits: the organization will gain a realistic view of its preparedness and key vendors will gain awareness of the specific requirements for recovery support.
The readiness review will also identify the detail requirements that need to be included in a requirement plan.
These requirements should involve all of the applicable components, including system software, applications hardware, logistics, transportation and other issues.
The review should also establish both the internal requirements and the requirements for each potential recovery site vendor.
The evaluation of recovery sites must therefore focus on ensuring they are the appropriate backups for unique recovery needs.
Once critical systems for recovery are determined and vendors are identified prior to evaluating recovery sites, companies will be in a better position to introduce potential site partners to their specific mix of in-house and third party applications.
At this point, it is beneficial to determine if any of the recovery sites are already familiar with the critical vendor-acquired applications. Certain sites may have prior experience with specific vendors and are therefore more prepared to handle system recovery. Such a three-way partnership could strengthen the recovery site operation during a declared disaster.
Each of the prime software vendors should be prepared to help document the responsibility of the recovery site.
In addition to the obvious requirements for properly backed-up data, software vendors should also help define: requirements for documentation; peripherals; communication links; all software (including operating systems, utilities and vendor applications programs); customized supplies; and personnel skill requirements.

Creating the
Written
Plan

Many organizations fail to create, complete or maintain a written plan, primarily to avoid the finality of possible disasters.
Support from internal users, recovery site vendors and software vendors will better ensure that a plan is completed and maintained.
Many plan preparation tools are available that include a methodology to keep the data processing organization, users and vendors working toward a common goal.
Share the chosen methodology with key vendors and encourage them to contribute to the plan building process. This will not only ease efforts during the most intensive phase of writing the plan, but will help involve and commit partners to the planning process.

Determining the
Extent of Vendor
Involvement

If the planning process is complete and the plan has been implemented and maintained, the likelihood of encountering a full-blown disaster is relatively remote.
But partial failures — which can be just as disruptive as massive failures — are more common as companies continue to distribute computing resources to remote locations, away from the protection of the raised floor, locked, air-conditioned data center.
The decision to invoke the plan for partial component failure or massive failure should and most likely will rest in the hands of an organization’s technology executives.
Once this decision is made, however, the team will move quickly according to plan to notify and involve all participants. At this stage, it is imperative that the team contact designated vendors to activate them.
While it is potentially disruptive to have multiple, well-meaning companies converge unnecessarily on a disaster site, vendors should be involved in the reconstruction of the application systems called for in the plan.
The nature of the disaster will determine if vendor involvement is best applied over the phone or onsite. Either way, software vendors should be made fully aware of the exact circumstances.
Since many vendors are originally involved in the installation and integration of applications and databases, it is not unreasonable to expect vendor support during hot-site implementation and production.
During this stage, companies go through most of the steps that occurred during the original installation and implementation of the software applications.
Software vendors may be more familiar with these procedures since their technical support staffs deal with them on a regular basis. Key vendors may also at this point serve as an audit function prior to the hot-site production process.

Assessing
Costs

Business recovery planning can represent a significant insurance policy but can also be very costly. A solid methodology in the planning process, therefore, must include a reasonable estimate of the costs involved.
In assessing costs, it is important to remember that vendors will likely charge consulting service fees for disaster planning services.
Vendors may draw on operations personnel, database administrators, programs, network specialists and customer support specialists.
While some of these services will be provided as part of the overall contractual relationship with the vendor, other vendors who become heavily involved in the recovery planning process will expect consulting compensation. Such compensation will provide a team of qualified individuals fully committed to disaster recovery planning.

Conclusion

Given the growing complexity of computer systems and the continued move to distribute operations, companies must involve vendors in the disaster recovery planning process to ensure the successful recovery of vendor-acquired software.
Vendors can and do provide knowledgeable support for disaster planning. They should be prepared to assist in keeping plans current through annual audits and be willing to provide technical support during declared emergencies and recovery site activities.
These vendor attributes can become key factors in selecting software systems during system evaluation and customer reference checks.
By following these steps, companies will be able to create a partner relationship with vendors who will be committed to quickly and efficiently resume operation of critical business applications when disaster strikes.

Ray L. Dicasali is senior vice president and chief information officer at Dun & Bradstreet Software (D&B Software). He is responsible for computer services, information systems, technical support, telecommunications and corporate services. He has worked in the applications software industry for nearly 20 years, closely associated with the development and implementation of a broad range of application software.


This article adapted from Vol. 4 #4.

Disaster Recovery Worldİ 1999, and Disaster Recovery Journalİ 1999, are copyrighted by Systems Support, Inc. All rights reserved. Reproduction in whole or part is prohibited without the express written permission form Systems Support, Inc.