New Orleans: Picking Up the IT PiecesBy David F. Carr | Posted 2006-03-06 Email Print
WEBINAR: On-demand webcast
Next-Generation Applications Require the Power and Performance of Next-Generation Workstations REGISTER >
City officials saw disaster preparedness as a job for another day. Then Katrina struck. In the six months since the hurricane blew apart the city, New Orleans officials have been improvising a plan to put its information infrastructure back together. Here
The day after Hurricane Katrina raked through New Orleans with 125 mph winds, a raiding party led by Greg Meffert, the city's technology chief, ventured out of the battered Hyatt hotel across the street from City Hall.
The streets were slowly but steadily filling with water from burst levees, and before long 80 percent of the city would be underwater.
Since the time the storm had hit, early on the morning of Monday, Aug. 29, Mayor Ray Nagin and his staff had been subsisting with practically no communications beyond an overloaded police radio network.
Cell phone towers had been knocked out, and most landline phones were dead, including those in the Hyatt. The satellite phones they'd brought into the hotel proved useless, after the phones' batteries malfunctioned.
As he headed out in a National Guard Humvee with his colleagues, Meffert had a plan to get Nagin back on the phone with the outside world so the mayor could call for help.
Earlier, the city's Web project manager, Scott Domke, had managed to establish an Internet connection off what was left of the Hyatt's network. That was a start, but Meffert needed more.
While the likelihood of a hurricane the size of Katrina hitting the New Orleans area had long been predicted, the threat was never deemed immediate.
A good disaster recovery plan, however, is well structured and rehearsed as if the event had taken placeMeffert's team, for example, could have done better by testing ahead of time the satellite phones they had been assigned, instead of finding out they didn't work when it was too late.
And while New Orleans did have an overall emergency response plan, which, for example, specified that the city would maintain lists of the phone numbers of critical personnel, the emergency plans placed too much faith in phone service remaining intact, Meffert says.
Residents trying to call 911 had their calls dropped after telecommunications switches dedicated to the emergency service failed.
Radio communications between police, fire and other emergency-response agencies were hampered by incompatible frequencies and equipment between state and local agencies, for example.
And although there was an emergency communications system that was supposed to bridge the radio frequencies used by state and local first-responders, it in turn was dependent on a landline telecommunications circuit that failed.
Meffert says his experience shows the weakness of relying on what he calls "hard-wired, 1950s technology" like traditional phone systems rather than more flexible Internet-based communications.
"The model still assumes that there will be one central number that everyone knows to call," he says.
Outside of a time of crisis, it's also easy to make unrealistic organizational assumptions, he explains, thinking that "even at this time of crisis, everyone's going to be sitting around the same table, stroking their beards, trying to decide what to do. That's not just wrong, kind of wrongthat's way wrong."
In reality, key players were scattered and often unable to talk with each other.
According to Meffert, the biggest mistake officials at every level made was treating disaster preparedness as a long-term problem that could be left for another day.
"Before this, the worst-case scenario always had sort of an ethereal quality to it," he says. "It was a virtual problem. And usually that gives you a virtual solution, rather than a real one."
In the six months since the hurricane struck, Baseline has been studying the role of technology in the crisis and recovery, spending time in New Orleans in November and January, interviewing city officials, and talking with technology and disaster recovery experts who have been assisting the city.
Much of what's admirable about the technology team's response was improvised, rather than planned. That's not surprising given the scale of the disaster, says Unisys disaster recovery specialist Ed Minyard, who assisted the city's recovery efforts. "Mike Tyson said everyone has a plan, and then you get punched," he says. "In the middle of something like this, 80 percent of what you think you know is questionable, and the other 20 percent is just dead wrong."