DRJ's Spring 2019

Conference & Exhibit

Attend The #1 BC/DR Event!

Winter Journal

Volume 31, Issue 4

Full Contents Now Available!

With 23 years of experience in software engineering Greg Povolny has developed technology for the Department of Defense, Pennsylvania's National Guard, and Florida's Department of Children and Families. He has seven patents for inventions with data interoperability. He is the founder of Mindshare Technology and the original architect of the SAMS technology and will use his experience and technology framework to deliver on state-wide solutions for emergency management and disaster preparedness.

Phelan: What do you see as the needs of the not-for-profit sector that can be met with IT solutions?

Povolny: From an IT perspective, the not-for-profit sector is left to its own devices with regard to emergency planning and disaster management. Often there is limited budget, lack of technology, limited or no standards and where technology solutions are deployed, they are almost always disconnected from other community partners. Today, with the far-reaching capability of the global Internet and its mass connectivity between stakeholders and partners, online information sharing and collaboration is a must and should be an integral part of the IT strategy.

Phelan: What is the significance of social networks, and do they play a part in creating synergies between the public, private, not-for-profit, and faith-based sectors for disaster preparedness?

Povolny: Social networks have been in formation for as long as people have been interacting. The more organized the social network becomes, and the more accessible the participants are, the more ready and the more proactive stakeholders can be daily, during and post disaster. Emergency support functions, the emergency operations center (EOC), and first responders have a good technology foundation. However, it is all too often disparate from each other. To compound the disparity, non-profits and faith-based organizations also are disconnected from such functions yet remain a critical support function long after the first responders and EOC have completed their tasks. The lack of collaboration and in particular, real-time collaboration makes the ability to organize infinitely more difficult.

Phelan: Where has Mindshare Technology made the greatest contribution in this market and/or in addressing such gaps?

Povolny: Similar to the emergency support function that coordinates and provides services within an EOC, non-profit and faith-based agencies also play a critical role in meeting the immediate needs and facilitating the long-term recovery of disaster affected citizens. The effectiveness of these efforts depends upon the extent to which these organizations, along with government agencies and other public and private entities, can share information in real time, coordinate essential activities, and communicate with persons in need. The absence of data collection architecture across disparate systems, in particular across the formal social services network of non-profit and faith-based organizations, is a significant problem, and threatens the operations of an emergency situation and jeopardizes the safety of all involved. These agencies must be able to work together and communicate situational awareness effortlessly to avoid a fundamental breakdown in delivery of social services before, during, and after a disaster. The SAMS approach to effortless information exchange is to automatically and proactively traverse the human relationships as defined in the social network, which is stored in the SAMS multi-dimensional participant relational database.

SAMS provides a self-defining approach to the natural evolution of community partnerships (social networks). There is inherent information sharing across such partnerships. It is our intention to facilitate the integration between our social network models. The SAMS service includes information sharing across databases that proliferate between community partners such as contacts, special needs, assets, victim registry, volunteers, realtime status, and dashboard solutions.

Phelan: What opportunities are there for business continuity planners in servicing this market?

Povolny: This is a great opportunity for business continuity planners in many ways. First and foremost, the use of such technology can be the core component to the communications plan. Furthermore, recovery plans can benefit through the real-time visualization of contacts, volunteers, assets, inventory, as well as a geographical depiction of situational awareness on a GIS map using local and state layers.

Mindshare is currently working with stakeholders in the City of Tampa and in particular with the community partnerships that interact with the Metropolitan Medical Response Services. We are excited to address the very real requirements that are facing our nation today. The ability to bring together all aspects of human interactions using the global Internet, collaboration technologies, and real-time, mission critical data sharing is here and now. Tom Phelan, Ed.D., is professor and program director, emergency and disaster management and fire science, at American Public University's School of Public Safety and Health.

The ongoing effort to create a more resilient society has moved to the forefront of disaster/emergency and business continuity management. This endeavor has never been more evident than through the recent activities of DRI International. Over the last few months, DRI International has helped facilitate a number of conversations with public sector entities to bring about a better understanding of goals and methodologies pertaining to private/public sector organizations. This has culminated in a joint public/private sector meetings held at the New York Stock Exchange on March 21, 2011. To supplement these efforts, DRI International has met and briefed senior administration and Congressional officials.
Untitled-3During these conversations it became apparent that there was a need for creating some basis of common understanding for public/private sector interfaces. Such understanding is critical considering that 85 percent of all resources used by the government are in the hands of the private sector. “The need to bridge the gaps and misconceptions is vital to both a government’s and a business’s ability to operate,” said Ira Tannenbaum, director of Public-Private Initiatives at NYC OEM.

The mission of emergency services and the delivery of governmental functions are duties and obligations of public sector employees. Often, the level of preparedness will have a direct effect upon life saving functions or a business’s ability to resume operations in the aftermath of an incident.

There has been much discussion in the continuity community of the parallel path travelled by emergency mangers, COOP professionals, and their private sector counterparts. Increasingly, it is acknowledged that there are many similarities, which when fully coordinated, work to achieve mutual benefits. That is not to say that there aren’t also some challenges to overcome.

In an attempt to create a convergent and coherent response to interruptions and to establish common ground for the union of private and public sector response, DRI International is developing training and certification programs specifically targeted to federal, state, and municipal public sector entities. This will be a step in bridging the terminology and procedural gap that now exists.

The course and certification, tentatively called “Certified Public Sector Resiliency Professional (CPSRP),” will provide a four-day course of study delineating the knowledge, skills, and abilities needed to keep government open during times of crisis. The class will review public sector terminology, laws, federal, state, and municipal guidances, including Homeland Security Presidential Directives. This class will offer best practices and lessons learned from major events over the past decades. It is important to understand that this is much more than an incident command class; it is an introduction into developing broad and strategic goals and methodologies to protect agencies, staff, facilities, and essential support functions. Government officials at all levels from federal, state, major cities and local jurisdictions will equally benefit from this training.

DRI International’s goal is to bring together the knowledge and experience of top resiliency professionals in the public sector with their private sector counterparts. By learning and sharing from best practices and lessons learned from colleagues with the same set of priorities, we grow as a profession. By being able to understand the many pressures experienced by disaster recovery professionals in the private sector and the providing of essential services by government, we can create a model for public/private sector resilience.

Alan Berman, MBCP, is a member of the ASIS BS25999 technical committee, a member of the Committee of Experts for ANSI-ANAB, a former member of the NY City Partnership for Security and Risk Management, executive director for Disaster Recovery Institute and the co-chair for the Alfred P. Sloan Foundation committee to create the new standard for the US Private Sector Preparedness Act (PL 110-53).
Is your disaster recovery plan set up for “virtual” success or a “real” nightmare? Traditional disaster recovery planning is critical to an organization but not easy to implement. In most production data centers today IT is forced to manage a mixed environment of heterogeneous hardware and software platforms. With business leaders focused on continuous improvement, IT managers are forced to tighten capital expenditures and contain costs while delivering a highly available, resilient, and agile IT infrastructure. In this highly demanding environment, IT staff must also demonstrate disaster recovery capabilities that meet the ever-changing business continuity metrics.

Why do so many traditional disaster recovery (DR) technology strategies under-perform and under-deliver? The simple answer is that a recovery plan using a physical server deployment that is equivalent in number to the production data center can be very complex. Delivering repeatable disaster recovery solutions with non-standard hardware and software platforms with already over-utilized human resources has never been more difficult.

The difficulty is exaggerated when you add additional business applications that require new underlying infrastructure. As companies adopt new business strategies, they often jump directly to a new technology initiative. While such moves often make business sense, they often don’t leave time for realizing the impact to the disaster recovery strategy. Traditional DR strategies require these changes are accompanied by planning, budgeting, and buying twice the required IT capacity. Yes, two of everything! It’s clear that continuous improvement directly impacts capital expenditures for traditional disaster recovery planning.

Traditional Barriers to DR Implementation

  1. Cost of solution. The biggest barrier to implementing a solid DR foundation is the prohibitive cost of matching production HW components; e.g. a one-to-one system ratio. This capital expenditure can be difficult to justify. Even when the benefits are clear, these are expensive solutions for any business.
  2. Infrastructure complexity. The number of systems to provision can consume valuable recovery time and funding. This is due in part to supporting numerous hardware components with dual maintenance challenges including the need for base metal restore.
  3. Reliability and repeatability of solution. Complex solutions are hard to test primarily due to the difficulty of provisioning sufficient equipment to re-create all servers that need testing. Because complex solutions are difficult to test, they are typically not tested frequently, which should leave the business questioning the reliability of the solution.
Relying only on a traditional physical server DR approach can prove to be out-of-reach for many cost-constrained businesses. In a many cases, it can lead a company to compromise on its disaster preparedness – a risky strategy. Instead, companies can chose a DR solution where the costs of implementation and delivery efforts reflect the intrinsic business value of the applications.

Can A Virtual DR Strategy Work For You?

The concept of server and storage virtualization is receiving broad support from enterprises in nearly all market segments. Most IT organizations are embracing virtual server infrastructure technologies as a means to cut costs in their production data centers. This technology is compelling in more than consolidation and server deployment activities; it also holds great promise for making DR a cost-effective and reliable deliverable. By implementing virtualization, you can minimize the number of physical servers needed for DR, which in turn allows for reduced complexity and cost by lowering power and rack space requirements. Virtualization can create significant improvements in the speed and simplicity of disaster recovery and is capable of further reducing recovery time objectives for mission critical applications.

Here are some underlying advantages that server virtualization provides when applied to meeting your disaster recovery requirements:
  1. Reduced direct costs. The biggest cost benefit comes from significantly reducing the quantity of physical production servers required for recovery. With virtualization, this becomes a many-to-one ratio. Consider anywhere from 10-40 virtual machines to one physical server ratio, depending on the type of workload.
  2. Effective hardware utilization. With virtualization, you avoid the large capital expenditures currently required to deploy under-utilized servers that lie idle in a traditional DR server setup.
  3. Standard hardware. All operating systems would see the same virtual hardware, allowing ease of migration between physical systems for maintenance or capacity management purposes.
  4. Standby operations. With physical to virtual (P2V) capabilities, virtual machines can be pre-provisioned with the operating system, application and system state ready for data to be restored.
  5. Easier testing/proven recovery. The simplified DR environment deployment equates to more frequent and thorough testing of the applications. Concluding a successful test, server configurations are maintained and ready for next use.
  6. Easier manageability. Recovery of your DR setup and delivery begins with key strokes and mouse clicks allowing for recovery from a remote location.
  7. Reduced indirect costs. In addition to space and power savings, there will be a meaningful reduction in needed human resources to deliver in a DR scenario.
  8. Handles tougher RPO, RTO. Additional capabilities exist to improve recovery objectives as business needs justify the additional expenditure.
There are, however, some disadvantages to consider with a virtualization solution:
  1. Learning curve to implement a new technology. Server, storage and network teams are impacted, creating a potential conflict in traditional administrative paradigms.
  2. Initial cost and maintenance of virtualization software and hardware. Server hardware and storage selected to meet the performance needs of the aggregate workload can be costly.
  3. Potential network upgrade. Network components will need to be reviewed to ensure sufficient bandwidth and connectivity are achieved for the virtual infrastructure. Upgrades may be necessary to accommodate aggregate workload performance demands.
  4. Larger impact with a server failure. Consolidation ratios of 10, 20, or even 40 virtual to physical servers increases the impact of a physical server failure. Careful planning should ensure workloads are distributed among physical servers, keeping application availability in mind; e.g. make sure not all domain controllers or web servers are on the same physical server.
  5. Requires production to be virtualized to realize full benefits. Full automation of system maintenance and recovery require production systems to be virtualized as well. Not having VMs in production requires mature and strict change control capabilities to ensure P2Vs are captured and provisioned accordingly. In large environments this can be highly burdensome on staff.
How A Virtual DR Scenario Works

In the past, physical server recovery required a complex and lengthy setup process (ghost, tape-based, bare metal recovery at the DR site), and separate hardware. With tape recovery, the DR process can be long, complicated, and error-prone. Finding the correct tape can also be a nightmare if cataloging is not up-to-date.

Virtualization dramatically simplifies the entire approach to managing your DR systems. You can now manage the OS and applications as a single unit. Traditional DR required changing over to new hardware, re-installing the operating system, followed by a process for installing and configuring the application. With virtualization, you can restore the OS and application as a single unit, more simply and much more efficiently.

Server virtualization breaks the dependencies between the operating system and the server hardware. The OS communicates with the “virtual hardware” provided by the software virtualization layer. The virtualization layer effectively makes the virtual machine hardware-independent. The virtual hardware seen by the operating system looks the same regardless of what x86 hardware is actually underneath the virtual machine.

Utilizing a physical-to-virtual (P2V) tool will help in creating virtual machines that are identical to their physical server equivalents. Virtualization software encapsulates an entire server into a set of files containing OS and data volumes as well as configuration files simplifying the components needed to recover a server. This set of files is the virtual machine (VM).

Virtual machines intrinsically possess two qualities that are beneficial to DR:
  • Hardware independence. Virtual machines are isolated from the underlying physical hardware. This characteristic of the virtualization makes it simpler to move VMs between physical systems.
  • Encapsulation. Virtual machines are encapsulated into a set of files.
Let’s look at the scenario in Figure 2 (above) where we restore physical servers to virtual machines.

In this example, we can use one single physical ESX server on the right to recover seven physical servers on the left. The back-up and recovery process remains the same as in a physical-to-physical scenario, and it still uses the same backup server and storage media that were used in a physical-to-physical server back-up scenario. When there’s a need to recover a system, you can use this archived virtual machine template and have it up and running in minutes, with applications and backup/recovery agent already installed. The great value of server virtualization is seen in how much faster and easier it is to stand up a recovered server. Once the virtual server is powered on, the data restore process will be identical to the restore process in the physical-to-physical scenario.

An increased benefit can be achieved by implementing the virtualization in production. The added benefits are:
  1. Production servers are already virtual (i.e.the VMDK file already exists, alleviating the need to perform a P2V every time a system change is made).
  2. Restoring a VM is equivalent to a complete system restore including data reducing RTO.

Traditional disaster recovery plans require many manual, complex steps to perform bare-metal hardware recovery. Virtualization simplifies this environment. Hardware configuration, firmware, operating system, and application installation become data stored in just a few files (VMDks) on your SAN. You can simply protect these files using your back-up or replication software, and you’ve protected the entire system. These files can then be recovered to any hardware without requiring any changes because virtual machines are hardware-independent. With hardware independence, you can repurpose existing servers for disaster recovery rather than needing to buy duplicate servers for DR. In this approach to recovering IT infrastructure, the focus moves from machines to a more holistic view that focuses on capacity, can be dynamically added or removed, and can be turned on instantly in the event of a disaster. So now you can have a “real” recovery rather than a “virtual” nightmare.

Richard Dolewski is chief technology officer and vice president of business continuity services for WTS. Dolewski is a certified systems integration specialist, disaster recovery planner, and is globally recognized as a subject matter expert for business continuity for IBM iSeries and i5 environments.

Click here for photos of Spring World 2011

Disaster Recovery Journal saw continued growth at Spring World 2011, March 27-30 at Walt Disney World’s Coronado Springs Resort.

“The show had a wonderful, positive feeling as it continues to grow back to the size before the economy woes,” said DRJ President Bob Arnold.

DRJ’s 44th conference saw another increase of nearly 100 attendees from last year, posting 920 registered attendees with more than 1,200 in total attendance.

“We had an incredible line-up of sessions and workshops,” said Arnold. “The attendees went away with a wealth of knowledge

“The feedback we received from everyone was incredibly positive. People were not only learning but doing so in a fun and enjoyable format.”

The expansive exhibit hall at Coronado Springs was also packed with cutting edge products and services to help attendees achieve the ultimate goal of creating a resilient organization.

DRJ Spring World 2011 Gold Sponsor Send Word Now sponsored a luau for the Monday Night Hospitality.

“Send Word Now hosted an extremely festive luau Monday evening,” said Arnold. “Attendees enjoyed authentic food, drinks, and traditional Hawaiian dancers.”

Other conference sponsors included COOP Systems, eBRP Solutions, Forsythe, IBM Business Continuity and Resiliency Services, Oracle, Recovery Point, Atlantic.Net, Booz Allen Hamilton, Emprimus LLC, MIR3, Strategic BCP, SunGard Availability Services, Consonus, EMC², Fusion Risk, Novell, Shermco Industries, SIOS, Verizon Wireless, Volo Recovery, xMatters, Attainium, Forrester Research, FedEx Custom Critical, Business Continuity Institute, DRII, and Public and Private Business Inc.

Matthew Paulson, CBCP, won the $500 attendance prize drawing while Angela Miele, CBCP, and Alison Phillips each won $250. All three attendees also won a free pass to a future DRJ conference.

“We are already deep into planning for the Fall World 2011 in beautiful San Diego. Hope to see you there. The theme for the Fall show will be Innovative Solutions Within Reach.”

DRJ’s Fall World 2011 will take place Sept. 11-14, in San Diego, Calif. For more information, see pages 67-71.

Jon Seals is an award-winning journalist and the editor in chief of Disaster Recovery Journal. Seals is a member of the Disaster Resource Guide Editorial Advisory Board and the Mid-America Contingency Forum.

Ryan Ault is the emergency response manager for Best Friends Animal Sanctuary in Kanab, Utah. He is also an armed forces veteran, a former lieutenant of animal services in California. Ault is certified in swift water rescue, rope rescue, large animal handling, trench rescue, collapsed structure, and wildland fire.

The "devil is in the details!" How many times have we heard this statement? Let's explore how this might apply to one aspect of business recovery planning.

Companies rely heavily on "critical staff" and have a tendency to feel they "will be there when needed." However, when a disaster strikes, people worry as much about their pets or livestock as they do their families. If there is little or no plan in place for this aspect of their lives, the company's critical staff will either not be available to help or won't be fully attentive to details the company might be expecting.

There are a variety of "not-for-profit" organizations that come together to assist with community disasters. Best Friends Animal Sanctuary in Kanab, Utah, has become internationally recognized for providing support to people and their beloved pets. The following is an interview with Emergency Response Manager Ryan Ault.

Williams: What can companies do pre-disaster to help reduce the need for your services in a disaster ultimately making the company and community recover quicker?

Ault: If businesses look at making sure their employees have the information or can get the education they need to be best prepared when a disaster strikes, the end results will be the business getting themselves back online and back to business that much faster. The less down time, the fewer losses! Small things business can do are provide/communicate the information that is already gathered for the animal owners and lovers among their employees from sources like Best Friends and FEMA. Make sure any issues or disaster specific to the businesses are talked about or have a link off their human resources site. Make the information easy and accessible to their employees and make sure they know about it. This is a win-win situation for businesses. It not only shows the employees that their concerns and well being are shared by the business, but the employees being better prepared will allow them to return to work and be productive that much quicker after a disaster.

Williams: What are your thoughts on the "lost or homeless pet" having consequences for business recovery if the pet was owned by one of their critical staff? If so, how companies can address this issue.

Ault: I believe that new hire orientations and further training for all employees can address how companies can help their employees with pet issues during a disaster and maybe inventory all company pets.

Williams: How do you prepare for responding to such a wide variety of community disasters?

Ault: We make sure our team is always training to prepare for what Mother Nature has in store for us. We also make sure we have a good working relationship with other agencies by establishing a memorandum of understanding pre-disaster.

Williams: Are you dependent on other agencies/companies to provide services and/or products before or during deployment?

Ault: While Best Friends is to solely dependent on any other agency, we are part of a coalition by the name of National Animal Rescue Sheltering Coalition.

Williams: Do you have a core team of staff dedicated to manage the full deployment process? What role do volunteers play in a disaster situation for you?

Ault: Our volunteers are the ligaments and tendons of an operation once the core team is deployed. The core team may be the "muscle" but without our volunteers to hold everything together (taking care of the animals rescued, supplying us with their expertise and skills that we don't have among the five of us) we aren't accomplishing anything really.

Williams: Is there an escalation process involved before you are deployed to assist?

Ault: Yes. We have to be deployed from a requesting party which is usually the state or agency in need. We then have internal procedures within our own agency. Once that is completed, a memorandum of understanding with the requesting party is always needed (if one is not already in place).


The relationship between humans and animals is very close and of primary importance. Animals are rescued and sent to a variety of locations so they are cared for and out of harm's way. These locations could be controlled by a variety of agencies. Capturing the names and contact information for potential agencies in your area could make this process quicker and less frightening for your staff and thus more available to your company. Minimally, companies in disaster-prone areas would benefit by factoring this into their recovery process.

For more information on Best Friends Animal Sanctuary and links to other animal rescue organizations, visit www.BestFriends.org.

Sara Williams, CBCP, is certified with DRI International. She is currently a business continuity consultant for Jack Henry & Associates. Williams recently rolled off the DRJ Editorial Advisory Board after four years of service.