DRJ's Spring 2019

Conference & Exhibit

Attend The #1 BC/DR Event!

Winter Journal

Volume 31, Issue 4

Full Contents Now Available!

The scenarios described are based on real events. Company names are withheld to protect security and confidentiality interests.

Put yourself for a moment in the CIO's chair at one of the nation's largest insurance firms.

From corporate headquarters in the Midwest, you manage information systems for a Top 10 insurance company with $60 billion dollars in assets, operating through 4,200 agencies across the country.

Your IS department proposes a new backup system: cancel your off-site data vaulting service and centralize server backups. In other words, it wants to funnel backups through your mainframe to an automated tape silo in the corporate data center. IS has what appears to be a sound business case for centralizing: automation, cost savings, and improved execution.

Foremost, says IS, the servers would be centrally administered and backups performed at one location, which would keep costs down. The silo itself would run as a "lights out" operation, so no need for extra personnel. The server backups would be near-line, so access would be automated and fast, providing quick and easy file recovery. And finally, the silo and the servers would not be in the same building. This allows the firm to discontinue their off-site vaulting contract, providing cost savings that would help offset the cost of the silo. Sounds good. So you authorize several million dollars and IT implements the centralization plan. But soon you learn the new setup has a huge, fatal flaw: It does not provide disaster recovery protection for the servers.

Why not? You notice first that the servers are indeed not located in the data center, but they are located in the building NEXT to the data center. Since the off-site vaulting service was canceled, the only set of server backups are right next door. So any natural disaster (flood, hurricane, tornado, etc.) could imperil both the servers AND the backups inside the silo. Your $60 billion firm is now vulnerable where it once was protected.

What's more, because the mainframe actually writes backup tapes, it writes them onto a mainframe tape format (i.e., 3480/3490). That's what the silo stores. These tapes cannot be mounted on a new server for recovery because the servers do not support that type of hardware. This means that even if the tapes inside the silo survive a disaster, you cannot use them to recover the servers. In that case, you'd need a functioning mainframe to rebuild the silo environment in which the backups were written. After that three-day, million-dollar process, then you could rebuild the servers.

Centralization Works,
If Done Right

As CIO, how would you react to this? You hadn't expected that centralizing your backups would introduce a threat to company survivability. But it's clear that IT didn't plan for a worst-case disaster scenario when developing the centralization plan. Remember, this is a real-life example. And in real life, this insurance firm is rethinking its centralization approach.

This is not an unusual story. Many large companies today are centralizing corporate server backups using a combination of software and hardware solutions. Centralization per se is worth considering; done wisely, it can provide cost and productivity benefits. But we've also seen that when carelessly approached, centralization can jeopardize a company's survival, by crippling its ability to recover from a real disaster.

Sure, convenience and saving money are attractive centralization benefits. But at what cost?

According to figures from Contingency Planning Research, every hour of data processing downtime at a national insurance firm can cost up to one million dollars in lost business. Add to that the costs of rebuilding data and systems. And even more chilling: the U.S. Bureau of Labor reports of companies experiencing a significant data loss, only seven percent are still in business after five years.

Can you have centralized server backups AND full disaster recovery abilities?

Yes, if you do it right.

To illustrate, let's look at another real-life example -- this time, a regional data processing center for a top-five U.S. bank with more than $250 billion in assets.

The bank has also centralized backups through its mainframe and writes them onto mainframe tapes. But in contrast to the insurance firm, it also writes backups on the servers themselves and sends these "native" backup tapes off-site.

Like the insurance firm, the bank still enjoys cost and execution benefits from centralization. But unlike the insurance firm, sending disaster recovery data off-site means the bank will recover if a real disaster strikes.

The bank doesn't rely on the silo to replace its off-site vaulting service. Generally, third-party off-site vaults are physically fortified to withstand catastrophic, natural disasters. They also prevent unauthorized access by external and internal threats. They are protected from fire and flood, monitored 24 hours a day, and equipped with professional media tracking and management systems. Outside the professional off-site vaulting industry, you'll find few companies possess those resources and capabilities in-house.

Make Sure Your Recovery Bases are Covered

Your company may already have centralized its server backups, or may be planning to do so. Either way, make sure the process does not compromise your company's ability to overcome a disaster. Here are a few key points to keep in mind:

  • Avoid the two-for-the-price-of-one approach

Don't let your file recovery system double as your fail-safe disaster recovery backup. If your only set of backups are kept on-line, they -- and your company -- are at risk. Period.

  • Provide true physical separation between backups and your data center

"Regional" disasters like earthquakes, floods or hurricanes can wreak devastation across miles. This is why "across campus" configurations do not provide true disaster recovery protection. And why many companies have minimum distance requirements (anywhere from five to 300 miles) for their off-site vaulting facilities. Compromising on physical separation for the sake of file recovery is a risky practice.

  • Allow for independent recovery

Make sure you also have server backups on tapes native to your servers, for example, Digital Linear Tapes (DLT) or 8MM tape. These native backups satisfy two key criteria for disaster recovery: for one, they can be easily transported off-site. Secondly, they permit an independent recovery -- that is, you can rebuild a lost server directly with these tapes. There is no need to rebuild a mainframe environment to identify and translate the backups.

  • Make sure you have off-line copies

Vaulting data off-site can ensure that a disaster to your data center will not put your company at risk. But even in the strongest facility, physical fortifications mean little if data remains on-line. This year's Information Week/Ernst & Young Information Security Survey of IT managers and professionals found that nearly 70 percent recognize computer terrorists as a threat. Equally important, more than 75 percent accept that the number one information security threat comes from employees and authorized users of their systems.

Once again, the key: Once data is backed up to tape, remove the tapes from any on-line access. Taking data off-line establishes a physical "firewall" to unauthorized access. For complete, fail-safe security, disaster recovery backups must be off-site, off-line and out of reach.

Kevin Koski is the VP/Technology at Data Base, Inc. (www.dbi.com ). Data Base, founded in 1976, provides off-site data security for 3,500 companies in 10 facilities nationwide. Koski can be reached at (800) 800-8110.