Business Continuity Chronicles

By Jeffrey M. Dato, MBCP

EDITOR’S NOTE: This is the fifth in a series of columns featuring the members of our executive council. Through these personal accounts, we hope to not only highlight their careers, but also give a 10-sided view of the history of the disaster recovery/business continuity industry.

After graduating from William and Mary in 1988, I was fortunate enough to land a job with Crestar Financial in Richmond, Va. It was there, working as an analyst in the trust operations group researching corporate actions (i.e. stock splits, mergers/acquisitions, dividends, class actions, etc.), that I first learned about the business world … and experienced my first taste of disaster recovery in action.
A large snowstorm hit the Mid-Atlantic, shutting down the most of the city’s operations and those of other cities on an arc north through Philadelphia. Since the markets in New York were not impacted, the Federal Reserve Bank (District 5) in Richmond and all the banks were required to be operational.
I walked several miles to work in nearly a foot of snow (my car had been sideswiped by a skidding bus earlier that morning), only to find my research tools – The New York Times, The Wall Street Journal and the Richmond Times-Dispatch (this was before the Internet was available to the general masses) – had not arrived due to the weather.
Without this data, I could not determine the potential impacts of the day’s planned “corporate actions” on our existing trust clients. With millions of dollars of our customer’s investments in the balance, I had to hold off on my research until the next day, not knowing the impact of this delay. Even if I could have done my research, our connection to the outsourced trust system (located in Philadelphia, where they had a failure of their systems due to the weather) had crashed. This event caused tremendous frustration within the department and left me wondering why we had not thought about this prior to then.
Around this time, Crestar became a nationally-chartered bank and began to fall under the auspices of the Office of the Comptroller of the Currency (OCC) and its recently-augmented regulation governing “disaster recovery” – Bank Circular 177, which was first mandated in 1983 – to include the concept of “business recovery.” Soon after the snow event, I began to experience “growing pains” and paid a visit to the personnel director.
She mentioned a several open positions (trust auditor, collections, call center) but none caught my attention. As I was preparing to walk out, she mentioned a newly-created position – disaster recovery analyst, an intriguing job title if I ever heard one. After a few job interviews with the head of risk management (my boss) and his chain of command, I was chosen from a field of 50 candidates to be the bank’s first contingency planner.
To this day, I am convinced the only reason I was chosen is that, at age 23, I was the cheapest resource to apply – an ascertation my former boss will neither confirm nor deny this assumption.
Crestar had been diligent from a technology front, having already written a “disaster recovery” plan for its data center operations, complete with a hot site agreement with Comdisco in New Jersey. What they had not covered, however, was everything relating to business operations.
Through my boss’s ties to insurance, I learned about the burgeoning disaster recovery industry. I knew I had a lot to learn, so I subscribed to two new magazines called Crisis and their competitor, Disaster Recovery Journal. From these periodicals, I began reading articles by Ed Devlin, Norm Harris, Jack Bannen and Rich Arnold about methodology basics in risk assessment, business impact analysis, hot site recovery, planning and testing.
I attended the first Delaware Valley Disaster Recovery Information Exchange Group’s seminar in Atlantic City (where we promptly experienced a horrendous Nor’easter, which threatened to cut off the island from New Jersey) and became a charter member of the Mid-Atlantic Disaster Recovery Association in the Baltimore/Washington metro area.
Most of the discussion of the time was based around the recovery of technology (mainframes, mid-ranges and 3270 “dumb” terminals). With my responsibility more focused on business, we cobbled together templates, a recovery organization and high-level policies to address the full-spectrum of contingency planning – response, recovery and resumption.
We teamed with facilities and security to assess internal and external risks facing each of our facilities. We conducted a crude business impact analysis to determine our recovery priorities, which we shared with the technology group so they could craft the hot site agreement accordingly. Reciprocal agreements for items processing, sorter operations and call centers were established with peer banks across town. With the advent of the first personal computers (and some basic knowledge with WordPerfect and dBase), we developed an archaic (yet effective) plan development application, where we built our first “business recovery” plans. We drilled the new plans with table-top exercises. We worked with our insurance carrier to ensure we had proper coverage where it made sense (i.e. “extra expense” instead of “business interruption”).
Comdisco’s new consulting practice visited, hoping to land advisory work. They were impressed with our program to the point that they called and asked if I wanted to join the practice. After a little arm-twisting, I was convinced to accept the offer and move to Bridgeport, N.J.
It was at Comdisco that I met some of the most talented professionals with whom I have ever worked. They had just landed a few deals with large banks and utilities where “business resumption” planning was required and needed a “non-techie” to assist with the delivery of the work. I jumped at the chance to both lead and learn. For the first time, I was exposed to a commercial integrated software program developed for contingency planning. This helped to standardize the planning process and simplified the replication of template plans across large organizations.
During my four years at Comdisco, the nation faced some of its most trying events, including Hurricanes Andrew and Bertha, the first World Trade Center bombing, the Midwest floods, the Chicago River floods and the Northridge earthquake. These “disasters” opened the eyes to most large businesses and forced them to realize the importance of its traditional technology recovery planning (now beginning to shift from mainframe/mid-range to personal computers and local area networks) and the new concept of business resumption planning, covering business processes.
After meeting the woman who would become my wife while at Comdisco in the mid-1990s, I made a “life” decision to stop traveling and begin a family. When Wynette accepted a job transfer to Richmond, I stumbled onto a job opportunity at Central Fidelity Bank, which had just become a nationally chartered bank (sound familiar?).
I was tasked with building their enterprise “business continuity” and “crisis management” programs and integrating them with the bank’s disaster recovery and information security programs. Additionally, the OCC became more demanding through more rigorous and aggressive requirements through Bank Circular 177 (which became the basis of the Federal Financial Institutions Examination Council’s Business Continuity Planning Handbook in 1997). Working within the risk management office, we were able to apply the “hot site” recovery and insurance coverage and claims adjustment lessons from my disaster support experience at Comdisco to strengthen our disaster recovery and insurance programs.
The disasters that had transpired since my days at Crestar helped to shape what would become our event and crisis management programs, taking lessons learned and building them into the program framework. We began utilizing business continuity planning software; expanding our commercial recovery contracts to include open or distributed systems, print and mail and items processing and progressed into “work area” recovery for our credit card operations; designing internal work area solutions for key business units; and working with local “first responders” to build a cohesive crisis and event management processes.
While at Central Fidelity, I was a founding member of the Business Recovery Association of Virginia, elected to the Disaster Recovery Journal Editorial Advisory Board and became a charter recipient of a new certification being awarded by the Disaster Recovery Institute (now DRI International) – the Master Business Continuity Professional (MBCP) designation.
Through my involvement with each of these organizations, I began to truly notice the difference in the discussion topics: “continuity” instead of “recovery,” “data availability” instead of “tape backup,” and “program” instead of “project.” Recovery time objectives (processes and technology) and recovery point objectives (data) were slowly moving toward smaller timeframes, which could not support the traditional alternatives in place for most organizations. The advent of a “quick ship” and “rapid recovery” solutions began to proliferate.
After the bank was acquired by Wachovia (and I had become a husband, homeowner and a father for the first time with a second on the way), I re-entered the consulting life. Since 1997, I have spent time at Ernst & Young, LLP; PricewaterhouseCoopers, LLP; and KPMG, LLP; where I am currently employed. Each stop has provided excellent insight into the inner-workings of all businesses (especially non-financial institutions) and additional business development responsibility, which I have grown to enjoy. During my tenure as a consultant, there have been extensive and unparallel changes in technology and the way business was conducted.
Remember how our industry was affected by Y2K, the Internet, technology expansion, dot-coms, denial of service attacks, globalization of business, outsourcing, Sept. 11, 2001, accounting scandals, layoffs, homeland security, and critical infrastructure?
Each of these events have had a significant impact on the business continuity industry and have pushed us to the brink of a new frontier: “resiliency.” Times are long gone when companies are willing to accept recovery times of 72 hours (or more) and once-a-week data backups – now it’s mirroring, spinning discs/OS, synchronous/asynchronous data pushes, data mining and RTOs and RPOs approaching zero. One would be hard-pressed to find an organization today, which claims to rely on no one – no supply chain or business partners or outsourcing processes/technologies, etc. Supply chain failure has been identified as is one of the top risks receiving focus by C-level executives. As a result of these events (directly or indirectly), regulations and standards across all industries and geographies have increased exponentially, focusing on areas around operational and technical risk, information security, privacy and business continuity. There is a progressive movement towards the concept of enterprise risk management, which encompasses compliance, financial, operational, strategic and technical risk – all areas that are touched by today’s business continuity professional.
Has the world of business continuity management, as it is currently known, changed much in the 16 years I have been involved in it? Absolutely! I anticipate it will continue to evolve just as rapidly over the next 16 years. To paraphrase a popular rock group known for its legion of committed and loyal fans, “…what a long, strange trip it’s been.”


Jeffrey M. Dato, MBCP, is a senior manager within the risk and advisory services practice of KPMG, LLP, where he is responsible for the business continuity management advisory practice for the Southeast and Caribbean.


 

©Copyright 2005 Systems Support Inc. All rights reserved. Reproduction in whole or in part in any form or medium without the express written permission of System Support Inc. is prohibited.

«BACK to the Articles Index