How To Guard Against

By Doug Bartholomew  |  Posted 2007-10-01 Email Print this article Print
 
 
 
 
 
 
 

When the data center at one of the country's biggest co-location facilities experienced an unprecedented power outage, bringing nearly half its customers down with it, the entire industry learned some painful but powerful lessons.

Massive System Failures">

GUARDING AGAINST MASSIVE SYSTEM FAILURES

Although the root cause of the outage is rare, the lessons gleaned from the experience are useful for data center managers seeking ways to guard against massive system failures:

  • Work closely with vendors: If 365 Main's technical staff had known the meaning of the error code or had had access to relevant online technical information, they'd have been able to solve the problem much faster.
  • Distribute data center resources: If you have the budget, establish a disaster recovery plan that allows for redundancy.
  • Communicate immediately and openly: "One of the things CIOs can learn from our experience on July 24th is that we communicated as transparently as possible," Kelly says. Sun's Snow agrees: "Their remediation and open correspondence on the problem and resolution was adequate."
  • Be ready to deal with the unexpected: "Sometimes in live environments things happen that you don't anticipate," Kelly says. "One hundred percent uptime is not a reality."
  • Never get complacent: "The extent of the disruption caused by 365 Main's power outage to the industry as a whole causes us to realize that, more than ever, a company can never be too prepared," Snow says. 365 MAIN AT A GLANCE

    Headquarters: 365 Main St., San Francisco, CA 94105

    Phone: (877) 365-6246

    URL: www.365main.com

    Business: Data center developer and operator providing mission-critical operations and business continuity for tenants

    Senior Vice President, Operations: Jean-Paul Balajadia

    Financials: NA (privately held)

    Challenge: Maintain the company's post-outage record of 99.9967% uptime across half a dozen data centers around the U.S., including the San Francisco facility

    • <123
     
     
     
     
    Doug Bartholomew is a career journalist who has covered information technology for more than 15 years. A former senior editor at IndustryWeek and InformationWeek, his freelance features have appeared in New York magazine and the Los Angeles Times Magazine. He has a B.S. in Journalism from Northwestern University.
     
     
     
     
     
    Manage your Newsletters: Login   Register My Newsletters