|
Business
Continuity Chronicles
By Jeffrey M. Dato, MBCP
EDITOR’S NOTE: This is the fifth in a series of columns featuring
the members of our executive council. Through these personal accounts,
we hope to not only highlight their careers, but also give a 10-sided
view of the history of the disaster recovery/business continuity industry.
After graduating from William and Mary in 1988, I was fortunate enough
to land a job with Crestar Financial in Richmond, Va. It was there,
working as an analyst in the trust operations group researching corporate
actions (i.e. stock splits, mergers/acquisitions, dividends, class actions,
etc.), that I first learned about the business world … and experienced
my first taste of disaster recovery in action.
A large snowstorm hit the Mid-Atlantic, shutting down the most of the
city’s operations and those of other cities on an arc north through
Philadelphia. Since the markets in New York were not impacted, the Federal
Reserve Bank (District 5) in Richmond and all the banks were required
to be operational.
I walked several miles to work in nearly a foot of snow (my car had
been sideswiped by a skidding bus earlier that morning), only to find
my research tools – The New York Times, The Wall Street Journal
and the Richmond Times-Dispatch (this was before the Internet was available
to the general masses) – had not arrived due to the weather.
Without this data, I could not determine the potential impacts of the
day’s planned “corporate actions” on our existing
trust clients. With millions of dollars of our customer’s investments
in the balance, I had to hold off on my research until the next day,
not knowing the impact of this delay. Even if I could have done my research,
our connection to the outsourced trust system (located in Philadelphia,
where they had a failure of their systems due to the weather) had crashed.
This event caused tremendous frustration within the department and left
me wondering why we had not thought about this prior to then.
Around this time, Crestar became a nationally-chartered bank and began
to fall under the auspices of the Office of the Comptroller of the Currency
(OCC) and its recently-augmented regulation governing “disaster
recovery” – Bank Circular 177, which was first mandated
in 1983 – to include the concept of “business recovery.”
Soon after the snow event, I began to experience “growing pains”
and paid a visit to the personnel director.
She mentioned a several open positions (trust auditor, collections,
call center) but none caught my attention. As I was preparing to walk
out, she mentioned a newly-created position – disaster recovery
analyst, an intriguing job title if I ever heard one. After a few job
interviews with the head of risk management (my boss) and his chain
of command, I was chosen from a field of 50 candidates to be the bank’s
first contingency planner.
To this day, I am convinced the only reason I was chosen is that, at
age 23, I was the cheapest resource to apply – an ascertation
my former boss will neither confirm nor deny this assumption.
Crestar had been diligent from a technology front, having already written
a “disaster recovery” plan for its data center operations,
complete with a hot site agreement with Comdisco in New Jersey. What
they had not covered, however, was everything relating to business operations.
Through my boss’s ties to insurance, I learned about the burgeoning
disaster recovery industry. I knew I had a lot to learn, so I subscribed
to two new magazines called Crisis and their competitor, Disaster Recovery
Journal. From these periodicals, I began reading articles by Ed Devlin,
Norm Harris, Jack Bannen and Rich Arnold about methodology basics in
risk assessment, business impact analysis, hot site recovery, planning
and testing.
I attended the first Delaware Valley Disaster Recovery Information Exchange
Group’s seminar in Atlantic City (where we promptly experienced
a horrendous Nor’easter, which threatened to cut off the island
from New Jersey) and became a charter member of the Mid-Atlantic Disaster
Recovery Association in the Baltimore/Washington metro area.
Most of the discussion of the time was based around the recovery of
technology (mainframes, mid-ranges and 3270 “dumb” terminals).
With my responsibility more focused on business, we cobbled together
templates, a recovery organization and high-level policies to address
the full-spectrum of contingency planning – response, recovery
and resumption.
We teamed with facilities and security to assess internal and external
risks facing each of our facilities. We conducted a crude business impact
analysis to determine our recovery priorities, which we shared with
the technology group so they could craft the hot site agreement accordingly.
Reciprocal agreements for items processing, sorter operations and call
centers were established with peer banks across town. With the advent
of the first personal computers (and some basic knowledge with WordPerfect
and dBase), we developed an archaic (yet effective) plan development
application, where we built our first “business recovery”
plans. We drilled the new plans with table-top exercises. We worked
with our insurance carrier to ensure we had proper coverage where it
made sense (i.e. “extra expense” instead of “business
interruption”).
Comdisco’s new consulting practice visited, hoping to land advisory
work. They were impressed with our program to the point that they called
and asked if I wanted to join the practice. After a little arm-twisting,
I was convinced to accept the offer and move to Bridgeport, N.J.
It was at Comdisco that I met some of the most talented professionals
with whom I have ever worked. They had just landed a few deals with
large banks and utilities where “business resumption” planning
was required and needed a “non-techie” to assist with the
delivery of the work. I jumped at the chance to both lead and learn.
For the first time, I was exposed to a commercial integrated software
program developed for contingency planning. This helped to standardize
the planning process and simplified the replication of template plans
across large organizations.
During my four years at Comdisco, the nation faced some of its most
trying events, including Hurricanes Andrew and Bertha, the first World
Trade Center bombing, the Midwest floods, the Chicago River floods and
the Northridge earthquake. These “disasters” opened the
eyes to most large businesses and forced them to realize the importance
of its traditional technology recovery planning (now beginning to shift
from mainframe/mid-range to personal computers and local area networks)
and the new concept of business resumption planning, covering business
processes.
After meeting the woman who would become my wife while at Comdisco in
the mid-1990s, I made a “life” decision to stop traveling
and begin a family. When Wynette accepted a job transfer to Richmond,
I stumbled onto a job opportunity at Central Fidelity Bank, which had
just become a nationally chartered bank (sound familiar?).
I was tasked with building their enterprise “business continuity”
and “crisis management” programs and integrating them with
the bank’s disaster recovery and information security programs.
Additionally, the OCC became more demanding through more rigorous and
aggressive requirements through Bank Circular 177 (which became the
basis of the Federal Financial Institutions Examination Council’s
Business Continuity Planning Handbook in 1997). Working within the risk
management office, we were able to apply the “hot site”
recovery and insurance coverage and claims adjustment lessons from my
disaster support experience at Comdisco to strengthen our disaster recovery
and insurance programs.
The disasters that had transpired since my days at Crestar helped to
shape what would become our event and crisis management programs, taking
lessons learned and building them into the program framework. We began
utilizing business continuity planning software; expanding our commercial
recovery contracts to include open or distributed systems, print and
mail and items processing and progressed into “work area”
recovery for our credit card operations; designing internal work area
solutions for key business units; and working with local “first
responders” to build a cohesive crisis and event management processes.
While at Central Fidelity, I was a founding member of the Business Recovery
Association of Virginia, elected to the Disaster Recovery Journal Editorial
Advisory Board and became a charter recipient of a new certification
being awarded by the Disaster Recovery Institute (now DRI International)
– the Master Business Continuity Professional (MBCP) designation.
Through my involvement with each of these organizations, I began to
truly notice the difference in the discussion topics: “continuity”
instead of “recovery,” “data availability” instead
of “tape backup,” and “program” instead of “project.”
Recovery time objectives (processes and technology) and recovery point
objectives (data) were slowly moving toward smaller timeframes, which
could not support the traditional alternatives in place for most organizations.
The advent of a “quick ship” and “rapid recovery”
solutions began to proliferate.
After the bank was acquired by Wachovia (and I had become a husband,
homeowner and a father for the first time with a second on the way),
I re-entered the consulting life. Since 1997, I have spent time at Ernst
& Young, LLP; PricewaterhouseCoopers, LLP; and KPMG, LLP; where
I am currently employed. Each stop has provided excellent insight into
the inner-workings of all businesses (especially non-financial institutions)
and additional business development responsibility, which I have grown
to enjoy. During my tenure as a consultant, there have been extensive
and unparallel changes in technology and the way business was conducted.
Remember how our industry was affected by Y2K, the Internet, technology
expansion, dot-coms, denial of service attacks, globalization of business,
outsourcing, Sept. 11, 2001, accounting scandals, layoffs, homeland
security, and critical infrastructure?
Each of these events have had a significant impact on the business continuity
industry and have pushed us to the brink of a new frontier: “resiliency.”
Times are long gone when companies are willing to accept recovery times
of 72 hours (or more) and once-a-week data backups – now it’s
mirroring, spinning discs/OS, synchronous/asynchronous data pushes,
data mining and RTOs and RPOs approaching zero. One would be hard-pressed
to find an organization today, which claims to rely on no one –
no supply chain or business partners or outsourcing processes/technologies,
etc. Supply chain failure has been identified as is one of the top risks
receiving focus by C-level executives. As a result of these events (directly
or indirectly), regulations and standards across all industries and
geographies have increased exponentially, focusing on areas around operational
and technical risk, information security, privacy and business continuity.
There is a progressive movement towards the concept of enterprise risk
management, which encompasses compliance, financial, operational, strategic
and technical risk – all areas that are touched by today’s
business continuity professional.
Has the world of business continuity management, as it is currently
known, changed much in the 16 years I have been involved in it? Absolutely!
I anticipate it will continue to evolve just as rapidly over the next
16 years. To paraphrase a popular rock group known for its legion of
committed and loyal fans, “…what a long, strange trip it’s
been.”
Jeffrey M. Dato, MBCP, is a senior manager within the risk and
advisory services practice of KPMG, LLP, where he is responsible for
the business continuity management advisory practice for the Southeast
and Caribbean.
©Copyright
2005 Systems Support Inc. All rights reserved. Reproduction in whole
or in part in any form or medium without the express written permission
of System Support Inc. is prohibited.
«BACK
to the Articles Index
|