Disaster Recovery – He Who Screams Loudest Doesn’t Recover First

Disaster recovery scenario: The servers are all down. The computer room is dark. A major disaster has occurred and you need to determine your next steps. What are your priorities? What task do you do first? In which order do you start your server recovery? Everything is a business priority, according to the business experts. Quick, lock the doors because a stampede of self proclaimed experts is about to come charging into the computer room and start barking out orders.

Are you going to listen to the person with the loudest bark and get his server back up and running first? If not, what IS your top priority? The computer systems may or may not be recoverable in the short term. Maybe they are not available for the long term either. You take a deep breath and tell yourself this is what we have been documenting and practicing for all these years. But does your current disaster recovery plan include prioritization of server recovery in a disaster?

Managing Mission Critical Servers for Business Continuity

There is a lot of work that goes into managing the on-going requirements for mission critical servers. When you have downtime, for whatever reason, data is unavailable to your customers, and this usually means that business – yours and your customers’ –simply stops. When business stops, it gets very expensive in a hurry. This is why critical server requirements should be reviewed twice a year to ensure that effective server processes are being carried out to support the true needs of the business and to ensure that these identified servers are still in alignment with business goals and priorities. Listed below are the elements that should be reviewed on a regular basis to support the critical server definition requirements.

• Business impact analysis and risk assessment
• Strategy for server recovery
• Change in prioritization based on different business cycles
• Application dependencies and interdependencies
• Application downtime considerations for planned and unplanned outages
• Backup procedures
• Offsite storage for vital records
• Data retention policies
• Recovery time objectives (RTO)
• Recovery point objectives (RPO )
• Hardware for critical server recovery
• Alternate recovery site selection
• IT and business management signoff

Classifying Systems for Disaster Recovery Priority

When you walk into the computer room it’s easy to be overwhelmed with rows and rows of servers. Numerous hardware platforms are powered on and ready to serve some business purpose. Typically you’ll find that the servers span several hardware generations. What’s required is a planned roadmap and prioritized recovery of your complete critical server infrastructure. You need to understand the supporting business needs of all servers in advance of any disaster ever occurring. Don’t wait for that phone call at 4 a.m. to decide your server recovery strategy. All the servers that reside in your computer room are not equal in level of importance to your business. That is why you need to consider the difference between what you need, what you want to have, and what you don’t need at all to run your business in a disaster.

The backup recovery team should assign priorities to the servers as they relate to your business support priorities. There will be a mixed bag of opinions, of course, but a good Business Impact Analysis will reveal which of those opinions carry the most weight. You should categorize the business requirements and supporting servers as Critical, Essential, Necessary, or Optional, as follows;

Critical Systems – Absolutely these servers must be in place for any business process to continue at all. These systems have a significant financial impact on the viability of your organization. Extended loss of these servers will cause a long term disruption to the business, and potentially cause legal and financial ramifications. These should be on the A-List of your disaster recovery strategy.

Essential Systems – These servers must be in place to support day-to-day operations and are typically integrated with Critical Systems. These systems play an important role in delivering your business solution. These should also be on the A-List recovery strategy.

Necessary Systems – These servers contribute to improved business operations and provide improved productivity for employees. However, they are not mandatory at a time of disaster. These might include business forecasting tools, reporting, or maybe improvement tools utilized by the business. In other words, minimal business or financial impact. The targeted systems can be easily restored as part of the B-List recovery strategy.

Optional Systems – These servers may or may not enhance the productivity of your organization. Optional systems may include test systems, archived or historical data, company Intranet and non-essential complementary products. These servers can be excluded from your recovery strategy.

These server classifications will provide you with the baseline for your decision making matrix. The key is your IT recovery team and your business management team must agree with the disaster recovery planning scope for classifications of the servers. By differentiating between critical, essential, necessary and optional, the reduction in the number of servers required to support the disaster recovery plan not only helps increase backup and recovery efficiency for the servers, but it also helps reduce your financial budget for disaster recovery.

The Big Picture

When compiling the list of mission critical applications, you must also consider application interdependencies. First, many software solutions are considered modular in design yet the software must be 100 percent intact — in other words, fully restored to function correctly. You cannot break the applications apart from the supporting infrastructure for the server. You may choose not to utilize specific business functions, but the entire solution must be rebuilt 100 percent to function normally.

Second, consider the flow of information. Follow the flow of a transaction from order inception to product delivery. You may find that a server not considered critical by the Business Impact Analysis does indeed have a significant role in feeding information back to yet another identified mission critical application. Therefore, IT input is needed in addition to the defined business needs. The restoration process for most servers is generally recovered in its entirety which includes every user library saved on the system. The question is, are you restoring too much? Omitting non-critical libraries can save hours, which translates to the business coming online more quickly in a disaster. The libraries and user directories that could be omitted include:

• Performances data
• Audit journals
• Test libraries
• ERP walk-through libraries
• Online education
• Developer libraries
• User test environments
• Data archives
• EDI successful transmission objects
• Trial software
• Temporary product work directories
• Auxiliary Storage Pools (ASP s)
• Independent Auxiliary Storage Pools (IASP )

Required Hardware for Your Disaster Recovery Plan

In the development of every disaster recovery plan, you must determine the minimum hardware requirements for your mission critical servers. Some IT professionals will say: “Obviously, you want your mission-critical servers to run the exact same equipment. However, in an emergency, any equipment is better than none. After all, it’s a disaster, not production.” This statement should not be accepted at face value. The reality is, only mission-critical applications absolutely need to be restored in a disaster, not everything. However, you will need to ask whether your business will accept running the “Mission Critical ” business functions at say 50 percent less capacity or throughput. In most cases, the answer will be no — totally unacceptable.

In the Business Impact Analysis you identified the financial impacts for your organization of being down for an extended period of time. Running your business at half speed will only further cripple your long term business capabilities and will not ensure customer satisfaction. Reduce the disaster recovery footprint by eliminating non-essential applications rather than providing less processing capabilities. Invest your disaster recovery budget wisely by supporting your business requirements in a disaster, and that means getting the right hardware. The last thing you want is your sales order desk telling customers to be patient; we can only process half the orders right now because we had a disaster and we are still working things out.