Business Continuity Takes Over for Disaster Recovery

Gone are the days of disaster recovery, when organizations had strategies in place to recover their data. Now, most healthcare organizations need to stay operational at all times so they are partnering with vendors to develop strategies. With a range of prices and scalable, flexible systems, there is no excuse for not adequately protecting your data.

Any organization with information management systems should be planning to not only safeguard that information but have a plan in place to make the information available during a disaster, says Jeff Pounds, executive director of IT at Baylor College School of Medicine in Houston, Texas, and speaker on the subject.

Overall, Pounds says healthcare organizations are doing well with disaster recovery planning. There has been a shift in thinking over the past few years, however, from disaster recovery to business continuity, he says. Disaster recovery traditionally meant that you back up all your information, bring your information systems down, ride the event out, and as soon as possible bring everything back up online. Rather than just thinking about how soon after a disaster they can get operational again, organizations are planning for remaining operational during the disaster. Hospitals are first responders during a disaster. “More and more, and particularly in healthcare organizations, you need levels of ongoing information during the event.”

Preparing to continue operating through a disaster includes identifying the categories of information that you do and don’t need during an event. Essential personnel require that certain information is available to continue to function during the event, Pounds says. “Some organizations determine that they don’t need much electronic information,” he says, “while others decide that they need as much as they can get.”


A slick setup

That business continuity idea applies to the strategy of Andrew Pipp, director of IT at the Center for Diagnostic Imaging, a group of freestanding imaging centers headquartered in Minneapolis. He started working with Acuo Technologies after another archive failed. “Our first archive decided to blow up on us. The database corrupted itself and because it was not a redundant product, all we had left was 1 terabyte of images on a tape jukebox.”

Acuo helped CDI migrate the old tapes to the current spinning solution. “Today, we have everything on spinning [EMC] Centera storage duplicating itself to another Centera storage device.” The goal in setting up an architecture with Acuo was ensuring that even a total disaster with one Centera unit would not bring down the network, Pipp says. By using the products to mirror images in real time on two separate infrastructures, Pipp hopes his customers won’t even notice any problems. All information is stored in the data center in Minneapolis and “dumb” devices are in each facility, forwarding images to the central location.

“It’s a pretty slick set up,” he says. “We don’t have to put much thought into our remote sites.” That includes not needing anyone to staff this at each facility and only needing one overall administrator. “That’s another thing we liked about [the Acuo product], it’s pretty self-sufficient. Once it is set up properly, it just works.”

New companies appear every year, Pipp says, offering “something bigger and better, but most companies’ products are in a proprietary format while Acuo does straight DICOM and keeps it open.” He recommends getting your plan down on paper first. “Understand how you’re routing,” he says. “If you don’t have that, it will cause major issues down the road.”


Being more proactive

Continuous computing is the goal at Cape Cod Hospital in Hyannis, Mass., says John Kilroy, vice president and CIO. “That’s a more proactive way to think about it,” he says. The facility worked with EMC Corporation to develop a storage area network (SAN). “We need to keep things running come hell or high water,” Kilroy says. High water could be a real concern since the hospital’s computer center is very close to Hyannis harbor which connects to the Atlantic Ocean. The strategy includes a redundant data center at another hospital 25 miles away, where the database is perfectly replicated as timely as possible.

It’s important to realize that you can phase in a business continuity strategy over time, says Pounds. “Don’t feel like you have to bite it all off at one time,” he says. “In many instances, people find a lot of lessons learned as they go through the process, so you can have more of an ongoing enhancement process.”

Incremental steps over the past five years have brought about the vision at Cape Cod Hospital. “None of this happened overnight,” says Kilroy. One important element was working with telecommunications carriers to make the bandwidth capabilities available and affordable. “We worked tirelessly to make that happen.”

The plan began with developing “more than the piece of paper that we and probably most hospitals had at the time,” says Jim Walsh, director of IT. “We had the vision then and went looking for technologies to fit that vision. That’s where EMC came in. From the beginning, we tried to come up with technologies that will allow us to bring one system back online at the same time as allowing us to have a fallback plan for a major disaster.”

The facility was already working with EMC for storage. For the continuous computing plan, “they didn’t have specific technologies that would work with our configurations,” says Walsh. But, “they had some potential answers to the questions that we had.” Along with changes occurring at the hospital, EMC’s software got better and better. “Finally, everything came together to put us where we are today,” he says.


Cost and communication

Facilities that aren’t doing enough in this area are probably facing “sticker shock” at the prices of solutions, Pounds says. But, the solutions Cape Cod Hospital is using have become available and affordable to facilities with a smaller size and budget over the past couple of years. “Just as PACS is now a reality in many community hospitals, this kind of plan is available to us now,” says Kilroy.

Pounds says he is often asked by IT and business administrators how they can communicate the need for business continuity planning to their senior leadership. Buy-in is important. “We’ve been setting the stage over the years of having a strategy of continuous computing,” says Kilroy. He and his team compared what would happen if everything went down before — what would have to be done and the time involved — with the situation with their vision in place.

“Our administration, much to its credit, has afforded us the ability to pay for the infrastructure to keep the computer systems up and running at a very high level,” says Kilroy. “It’s insurance and we’ve had to sell it that way. We have not presented this as a continuous computing plan just to an IT steering committee. We have been educating all of senior management and regularly keep them apprised of our plans.” Kilroy also put the IS infrastructure in a different bucket than applications projects so that they aren’t seen as competing for resources. “That has worked well for us.”


Reconsider and reassess

Another aspect of planning is revisiting your systems for reassessment. “Organizations are dynamic, they change,” Pounds says. “It’s always good to reassess your business requirements and whether you are meeting those requirements.” He recommends conducting an analysis to find any gaps. “You’ve got to be careful and not get too comfortable with having a particular program in place that just sits on the shelf.” If you’re not invoking your disaster recovery/business continuity plan until you have an event, perhaps not for years, you very well may not be adequately prepared for the event. “Pay attention to what is changing with your business to ensure that you’re in a position to make information available,” Pounds says.

The business changes that came with implementing PACS led Mike Hopkins, radiology director at St. Elizabeth Regional Medical Center in Lincoln, Neb., to reconsider storage. He started working with InSite One primarily for long-term storage. “Our onsite storage was filling up and we were looking for a cost-effective way to store images that were a year old or so.” One of the benefits of InSite One, he says, is that “our images are stored in two different places and so creates a DR plan.”

Shorter-term images are stored both onsite and at both of Insite One’s secure locations. Hopkins is comfortable with the set up because “if both sites that InSite One has are not operational and we’re not operational, retrieving images is not the primary concern.”

Hopkins drove this project from the radiology department but didn’t appreciate the security aspect until Hurricane Katrina hit. At the time, it seemed to be a value-added offering. Natural disasters plus the increased use of PACS at mid-sized facilities has driven awareness of disaster recovery planning, he says. Three to four years ago, people were still relying on film for long-term retrievals because PACS was still young. “With film, everybody knew it could only be in one place at a time and we did the best we could to protect against things like fires,” he says. “But when there’s something more you can do and you don’t do it, you really open yourself to criticism. It’s real easy to set up a disaster recovery place where you send your images. Cost is not a good excuse.”


Protection priorities changing

These concepts become particularly important as the majority of healthcare organizations move away from the paper world and into the digital world, says Pounds. Now, paper supports electronic information rather than electronic data supporting paper information. “As we continue in healthcare to get more and more electronic, we’re going to have to pay very close attention to disaster recovery and business continuity and how we’re going to protect that information,” he says. “We’re going to find more and more that the business continuity concept is much more important than what we traditionally think of as disaster recovery.”