ZIFFPAGE TITLELessons Learned

By Anna Maria Virzi  |  Posted 2006-09-07 Email Print this article Print
 
 
 
 
 
 
 

Dennis O'Donovan, director of New York information services at law firm Sidley Austin, was working in the firm's World Trade Center north tower office on Sept. 11, 2001, when the first hijacked plane smashed into his building.


Lessons Learned

Q: How has your organization's business continuity plan changed since 2001?

A: We increased our emphasis on remote access. In a building-down or -gone scenario, it may not be possible to get contiguous space. So the focus is more on putting up a data center and granting access to either satellite offices or even home, using standard remote access.

Q: What were other lessons learned?

A: Diversifying our carrier feeds—data carriers, Internet providers, wide area network providers and voice carriers.

Q: Who are the carriers you are using now in New York?

A: For data, we have AT&T, Yipes and Cogent. On the voice side, we have Verizon and AT&T. And we have MCI as the "local loop" provider for some of our AT&T voice circuits. We are started to implement on a limited basis, Voice over IP, so our wide area circuits can carry voice.

Q: Why is it important to have multiple providers?

A: There are a lot of different flavors of disaster and emergency events. For instance, when we had the blackout in 2003, Verizon cell phones did not work. But Blackberry cell phones did. That diversity ensures that there is some level of [service].

Q: Having gone through the experience you did, what three or four pieces of advice would you give to other CIOs preparing or updating their business continuity plan?

A:

  • Format your disaster recovery plan or continuation plan in the form of instructions, keeping it practical, step by step and short and sweet. Write it to be used. A lot of people write wonderful things that look good on a shelf, but are never going to be used.
  • Test and update your plan. We have been rigorous about that. Every time we change a back up method or procedure, that gets documented. Personnel changes are constant. It gets useless without testing. We find opportunities when we do maintenance to do DR [disaster recovery] testing as well.
  • Maintain meticulous records of inventory-- hardware and software, and also your backups. We found that having clean inventory records significantly facilitated the cash flow from insurance companies, which was a critical factor in how quickly we recovered. We had to have necessary cash flow in addition to our credit resources to turn around the materials we had to bring onsite.
  • Document and plan for people's roles and channels of communication. We found that everybody knew generally knew who to contact and how. As we learned with the 2003 blackout, sometimes those systems break down. We made a significant effort to make all end users aware of where they should call for the most up-to-date information and instructions--i.e., Is the office going to be open or closed? People have one phone number to call for information. Conference bridges are set up in advance that technology and administrative people can rally in virtual meeting rooms to plan out recovery.

    Q: So some lessons came out of the blackout. What did you learn from that experience?

    A: What we learned involved clarifying roles. The majority of the I.T. department knew that when there is a disaster, it is all hands on deck. We are a part of the life and building safety team. With 9/11, all of the disaster recovery training typically focuses on technology but always pays lip service to the human factor. For the first 48 hours, the human factor was much more in the forefront for everybody. One of the things I did in the first 48 hours after 9/11 was to literally call up every single employee of the downtown office and the midtown office to make sure every last person was accounted for. As a member of the managerial team, we needed as many people as possible to do that.



  • <123
     
     
     
     
    Executive Editor
    avirzi@ziffdavisenterprise.com
    Anna Maria was assistant managing editor Forbes.com. She held the posts of news editor and executive editor at Internet World magazine and was city editor and Washington correspondent for the Connecticut Post, a daily newspaper in Bridgeport. Anna Maria has a B.A. from the University of Rhode Island.
     
     
     
     
     
     

    Submit a Comment

    Loading Comments...
    Manage your Newsletters: Login   Register My Newsletters