Your Disaster Recovery Plan
By Joe Dishon, Syndicate Sales, Inc.
Do you need one? One word…YES! If you are in business today and want to remain in business following a major disaster, then you need to have a plan. What is it? A disaster recovery plan is exactly that. It is a plan that is developed and routinely maintained that documents all of the information you would need to recover from a major disaster.
Have you ever asked yourself what would happen if you had a fire and it destroyed your facility? To put your head in the sand and say, “it will never happen to me” is not a good answer. Ask the folks from New Orleans about Katrina, the people from Joplin about the tornado, or the guy down the street whose business burned to the ground. I am sure that they didn’t think it would happen to them either. Some people will say, “I have insurance, I am covered”. Well, that’s a start, but a Disaster Recovery Plan does so much more than that.
A well developed Disaster Recovery Plan will take you step by step through the process of getting your business back up and running. The plan will contain lists of key contacts, employees, vendors, customers, etc. You will have your equipment listed with all required specs, sources of supply, etc. You will have these lists not only company wide, but you will break it down into individual departments as well. It will contain both short term and long term plans. The short term plan is to get you back up and running in some capacity. The long term plan is to get you back to full strength. The most important piece is maintaining the plan. As your business changes, so does your plan. If you replace a piece of equipment, it is important that you document that change in your plan.
The area of the plan that we are going to concentrate on in this document is your data and IT infrastructure. While equipment can easily be replaced, data cannot. So it is vital that you protect your data and have a plan in place to get it restored and running as if no disaster occurred. As you put this plan together, you really can’t over document. It is better to have more information than you need than to be missing some key items.
Lists are an important part of the plan. They give you a reference to refer to as you begin to recover from the disaster. Following are examples of the types of lists you want to include:
Tech Support Contacts – This is a list of all of your vendors with contact names, phone numbers, email addresses, account numbers, and a description of service provided
Network Equipment List – This contains all of your devices (servers, routers, switches, etc.), IP addresses for each, user name, and description of device function
DNS Documentation – This contains your domains and their IP addresses
IP Address – Contains a listing of all IP addresses (internal, external, DMZ)
Licensing – This contains all of your software licensing. It will have the name, product key, number of seats, and description of the program
Equipment Configurations – contains the documented setup of each device (router, switch, server, pc)
Backup Schedule – contains the schedule you were using to backup your data
Users – contains a list of users and access
Computers – contains all of your computers, user name, software included on machine, and specs of machine
Passwords – Many of the above lists require passwords. For security purposes it is not a good idea to document the passwords right on the same document. This needs to be kept very secure. Password encryption is highly recommended.
Protecting your data can be done in a number of different ways. As technology continues to advance, so do your options of protecting your data. Only a few short years ago, your only option was to back up to tape which was always done in a batch process. Backing up in batch meant that you were performing a complete data backup once or twice a day. If you had a problem, you would lose any data between now and the last backup. There is a newer technology (De-duplication) that allows you to backup more often by only backing up the data that has changed since the last backup. This lessens the potential amount of data loss. You also have the option to do real-time data replication. This option eliminates the loss of data and can potentially speed up the recovery process.
The most traditional way to protect your data is to back it up to tape. Backing up to tape is typically done daily, month-end, and year-end. This is still a very viable solution.
Important Note: These tapes must be stored offsite.
The second way to protect your data is to back up to disk. This is done basically the same way you back up to tape but you are using disks instead of tapes. An added feature provided with this technique is that you can perform this backup online to another facility and eliminate the need to move the disks offsite. The only drawback to this technique is that depending on the speed of your internet connection, you are limited in the size of the data you are backing up.
This process is fairly new but takes the batch process to a new level. De-duplication allows you to backup only the data that has changed. This backup is merged with your initial full backup to create a complete backup of your data. The big advantage here is that you can backup your data more often and complete the task in less time. Also, by only backing up the data that has changed, it makes backing up over the internet to another site much easier.
Real-Time Data Replication
There is software available now that allows for real-time data replication. As you perform a transaction in your live database, that same transaction is being duplicated in your backup database. Again this is preferably done over the internet to another site. The big advantage here is that you can failover to the duplicate database in a very short period of time. This could even be used in the case of a server failure to your main live database.
It is a good idea to have several copies of all of your software programs. You should keep these copies in secure locations. Of course you want to have a copy at your main site, but you will want to keep at least one copy off site. This will make the recovery go much smoother.
Disaster Recovery Site
You have several options here. First, you may decide to only take data offsite. Second, you can maintain your own site. Last, you can use an outside agency to host a site for you. Your decision here can be determined by a number of things, primarily your budget. A disaster recovery plan is really an insurance policy in case you ever need it. How much you invest really depends on how quickly you need to be back up and running. Depending on your current infrastructure, one option may be much easier to implement than another.
This is by far the most inexpensive option with regards to upfront costs. By taking your data offsite you are fulfilling the biggest task which is protecting your data. What you are giving up is recovery time. You have your data, you have a copy of your software, and you have your DR plan. What you don’t have is any equipment or infrastructure. Your downtime is going to be determined by the time it takes you to purchase new equipment and get your data restored and running. If you are a single location, this may be a good fit. If you have multiple locations that connect to a central server, this could be a disaster.
Host Your Own
This is a great option for companies with multiple locations that share a common database. Putting your DR site in one of your remote locations protects you from having a disaster in one location from affecting all of your locations. Using this option along with Real-Time data Replication can help you keep running in the face of a disaster.
There are a large number of companies that specialize in Disaster Recovery. They can do as much or as little as you want. They can range from holding your backup tapes to maintaining servers for you to use in the case of a disaster. They even have the option of putting a mobile office on site for your employees to use while rebuilding your facility.
If you are using a hosted site today for your IT needs, they probably offer these services as well. It is important that you ask them about their DR plan. If they don’t have one, they may not be the right choice.
Internet access is your life line to the outside world. Without it, you can’t send or receive emails, your customers can’t access you website, and any of your branch locations cannot connect to your system. This is also one of the more complicated pieces to your disaster recovery plan.
In most cases, your internet provider has built a lot of redundancy into their system. If you are currently being serviced by a hub in Chicago and it goes down, they will automatically re-route you through another hub. There is a real good chance that you will not even know that they re-routed you. But what happens if your access gets cut off right at your location? If this happens, you will be down until it gets fixed. Unfortunately, an emergency for you may not be an emergency for them. So how do your cover your bases? Following are a few options.
This technology is available. This would be a second internet source that you could use in the case of your primary source going down. There are several issues to be concerned with here. First, you are paying for a second internet connection. Second, while there is technology to help replicate your DNS to the new internet source, you can still experience some down time while full DNS replication takes place. Experts say this could take up to 48 hours, but in most cases is completed within the first couple of hours.
Second Internet Vendor
This would be the same as the satellite feature, but it would be a land line. There is an additional issue to be concerned with here. The second internet connection is more than likely traveling on the same line as your primary. If your primary internet source is lost, so is your second. You should verify that this is not the case before selecting this option. The other option here is to connect this second source to your DR site. Once you bring up your DR site, and replicate your DNS, you would be back up and running.
MPLS – Multiprotocol Label Switching
This technology is fairly new and until just recently was very expensive. Like most technologies that have been out for a while, the cost is coming down to make this an affordable option. This allows you to have internet and ethernet connections to all of your locations including your DR site. Each location has the ability to send and receive data to the internet or any of your other locations without connecting to your central location. (For single locations, the only other connection would be to your DR site.) By connecting your DR site to this network, as soon as your bring up your DR site, all of your locations are back up and running. This option does not require DNS replication.
Your plan is a documented resource for you in the case of a disaster. It is important that it contain the above mentioned details. Most importantly, it should contain a step by step process for recovery. Be as detailed as you can. Remember, if a disaster occurs, you may not be thinking as clearly as you are today. Every detail counts. Maintain the plan. Maintain the plan.
Periodic reviews are extremely important. It is a must that you keep your plan current with any changes. The best way to test your plan is to do a mock test. Get the right people in the room and act as if a disaster just occurred. Go through the plan on a quarterly basis, you may be amazed at what you missed or what you will find has changed.
You may never experience a disaster, but if you do, you will be glad you have a plan in place to help you recover.