WinterBy Mel Duvall | Posted 2007-04-05 Email Print
Know the Risk: Digital Transformation's Impact on Your Business-Critical Applications REGISTER >
A Valentine's day ice storm put a strain on JetBlue's systems and processes for handling reservations, rerouting traffic and tracking flight crews. How did they handle it?'s Embrace">
The weather front that led up to the Valentine's Day storm had already been causing trouble in the Midwest for several days before it reached the East Coast. But while it primarily delivered blowing snow and frigid temperatures in cities like Chicago, the storm had a different cargo by the time it reached New York. Freezing rain and sleet slammed the mid-Atlantic region, causing havoc on major highways and virtually shutting down airports.
While most other airlines cancelled dozens of flights in preparation for the storm, JetBlue management opted to wait it out. The airline's policy is do to whatever it can to ensure a flight is completed, even if it means waiting for several hours. According to chief executive officer David Neeleman, the airline sent outbound flights to the runway at JFK at about 8 a.m., to be ready to take off as soon as the weather let up, while incoming flights arrived and filled up the gates. But instead of improving, the freezing rain and sleet continued. Under federal aviation guidelines, planes cannot take off in ice pellet conditions. Soon, planes and equipment were literally freezing to the tarmac.
By 3 p.m., the airline gave up hope of getting the planes sitting on the runway off the ground and began calling in buses to bring passengers back to the terminal. But by then, the damage was done. Airport terminals, particularly the JFK hub, were filled with passengers still expecting to get on flights and they were now being joined by hundreds of infuriated passengers getting off planes. "Things spiraled out of control. We did a horrible job," said Neeleman in a conference call early the next week. "We got ourselves into a situation where we were doing rolling cancellations instead of a massive cancellation. Communications broke down, we weren't able to reach out to passengers and they continued to arrive at the airports ... it had a cascading effect."
In fact, the impact of the Valentine's Day storm would last for almost a week as JetBlue attempted to accommodate passengers on planes that were already nearly full. In response, Neeleman introduced a JetBlue Customer Bill of Rights offering various forms of compensation to customers whose flights have been cancelled or are left sitting too long on planes. The direct financial impact was estimated by Neeleman to be as high as $30 million, although he says it's the blow to the company's reputation that stings the most.
During his conference call with investors, Neeleman made a point of stating that JetBlue's computer systems were not to blame for the meltdown. "It's important to note our software providers didn't fail," he said. While Mees agrees with that assessment, he also admits that there were a number of shortcomings in the company's information systems. In some instances, such as the reservation system, the airline was not able to expand the system enough to meet the extreme customer call volume, while in other instances, such as keeping track of off-duty flight crews and lost baggage, there simply weren't systems in place.
As the seriousness of the situation began to unfold on Valentine's Day, managers with JetBlue's Salt Lake City-based reservation office began calling in off-duty agents to assist with the expected high volume of calls. JetBlue reservation agents primarily work from their homes, using an Internet-based communications system to tap into the company's Navitaire Open Skies reservation system. Navitaire, headquartered in Minneapolis, hosts the reservation system for JetBlue as well as about a dozen other discount airlines.
As passengers were bumped off planes or arrived at JFK and other East Coast airports to find their JetBlue flights had been cancelled, they had only one option to rebook their flights: call the JetBlue reservation office. JetBlue does not offer its customers the option to rebook their flights via its Web site, nor can passengers rebook using airport kiosks. According to Mees, the option of rebooking via JetBlue.com is something the company began working on in December 2006, before the February disruption. But the new feature was not yet ready to be rolled out. As a result, the Salt Lake City reservation agents were suddenly deluged with calls from irate passengers looking to get on another flight or find out what compensation was available.
The Navitaire reservation system was configured for JetBlue to only accommodate up to 650 agents at one time, a number that more than met its requirements under normal circumstances. As customers' wait times on phones ballooned past an hour—if they could get through at all—Mees put an urgent call through to Navitaire to see if anything could be done to increase the number of agents using the system. Navitaire was able to boost the system to accommodate up to 950 agents at one time, but then it hit a wall. More agents could not be added without impacting system performance. Mees says Navitaire is working on an upgrade to the system that will allow it to expand to substantially higher numbers, but for now it has reached its limit.
Baseline contacted Navitaire for comment, but it referred questions about timing and capacity issues back to JetBlue.
Even with the number of agents that could be accommodated expanded to 950, JetBlue was having difficulty finding enough bodies to staff the phones. It employs a total of about 1,500 agents. Off-duty crews and airport personnel volunteered to staff phones, but they were not trained in how to use the system. Such system cross-training is not common at most airlines, but it's something Neeleman vows will become common practice at JetBlue in the future. He says management and certain staff segments will be cross-trained on such functions as reservations and crew scheduling to bolster ranks during an emergency.
As passengers struggled to get through to reservations, their bags piled up in huge mounds at airports, particularly at the airline's hub at JFK. Surprisingly, JetBlue did not have a computerized system in place for recording and tracking lost bags. In 2003, it entered into an agreement to purchase a system from Lufthansa called BagScan, but the system was never implemented, Mees says. "We didn't prioritize it—probably because we were so focused on the SAP project," he admits. JetBlue was growing quickly and needed the increased capabilities in the SAP ERP system to handle its human-resources functions.
JetBlue has a data warehouse that stores reservation and check-in information, such as the number of bags checked in by a passenger and the bag tag identification numbers. What was missing was an add-on component to record which bags had not been picked up and their location. There was no way for a passenger agent, for example, to look up by computer if a lost bag for a particular passenger was among the heap of unclaimed bags in New York. Not having this functionality had never been a big problem in the past, Mees says. If there were bags left over at the end of a flight, airport personnel figured out ownership by looking up a passenger record. Because JetBlue rarely cancels flights, the process had been manageable.
Not this time. "There were so many bags, they went from floor to ceiling—and where there weren't bags, there were people," Mees says. A technology team (JetBlue has 193 people in its I.T. department) was dispatched to the airport to help out with the effort. They ended up hauling most of the bags to an offsite location where the bags could be sorted and identified. Over three days, programmers cobbled together an application using a Microsoft SQL database and handheld devices from Symbol Technologies that could scan a bag tag and identify the passenger. Agents could then access the database to provide passengers with information on the location of their lost luggage.
Meanwhile, in flight operations, managers were attempting to sort out a number of other headaches. In addition to the Navitaire reservation system, JetBlue uses several applications from Sabre Airline Solutions of Southlake, Texas, as part of its core operations infrastructure. The Sabre Flight Control Suite provides the airline with applications to manage, schedule and track its planes and crews, while Sabre's Dispatch Manager application is used to develop the actual flight plans. Sabre's FliteTrac application, part of the Flight Control Suite, interfaces with the Navitaire reservation system and provides managers with real-time information on factors such as flight status, fuel information, passenger lists, and the original, revised, estimated and actual arrival times. The Sabre CrewTrac application, also part of the Flight Control Suite, tracks crew assignments, ensures legal requirements are met, and provides pilots and flight attendants with access to their schedules via a secure Web portal.
JetBlue also utilizes a sophisticated system from Navitaire called SkySolver to help figure out the best way to emerge from flight disruptions. SkySolver allows operations planners to plug in a number of scenarios, such as canceling flights or redeploying planes and crews, to figure out which actions will get operations back on track the quickest while minimizing passenger disruptions. It might, for example, recommend a plane traveling from Burlington, Vt., to Orlando, Fla., with a stopover in New York, fly directly to Orlando if that is the final destination of most passengers. JetBlue has used SkySolver in the past to help it during major disruptions, including the 2004 hurricane season.
Mees says early on during the disruptions on Valentine's Day, planners worked out a number of scenarios using SkySolver to get their operations back on track. However, when they attempted to transfer the solutions—or "solvers," as they are called—from SkySolver into the company's Sabre applications to produce flight plans that could be quickly acted upon, SkySolver was unable to transfer the information into Sabre. According to Mees, Navitaire immediately tackled the problem and was able to come up with a fix in a matter of hours, but by the time the problem was fixed, "the damage was already done." Gates were full, flights could not be cancelled in an orderly fashion, and the airline virtually reached gridlock.
Mees says he doesn't know why the information could not be transferred or why the glitch was not discovered beforehand. Navitaire would not provide a response to Baseline's questions. For now, Mees says he has other priorities to work on and was satisfied with Navitaire's quick response.
Even if JetBlue had been able to work out a game plan to bring a quick end to the disruption, there was no guarantee it could have gotten flight crews to the redirected planes. In the days following Valentine's Day, JetBlue struggled to keep an accurate account of its pilots and flight crews. Normally, during disruptions, off-duty crews call into headquarters to give their location and availability to work. But with the sheer number of flights affected—more than 1,000 were cancelled over the three-day weekend for Presidents Day—phone lines were busy, calls couldn't get through, and information had to be cobbled together with pen and paper. JetBlue did not have a database system to keep track of off-duty crews in such a situation.
Once again, the technology team created a simple, centralized database over a 24-hour period to keep track of crew locations and their contact information, using a Microsoft SQL server database and .NET tools. Mees says he will keep that system for the interim, but the plan is to develop a more full-featured system that might allow pilots and crews, for example, to plug in their location and availability via the Internet or a handheld device.
It's just one of a number of initiatives that have suddenly vaulted to the top of Mees' priority list. Others include:
By early April, work with Navitaire to add a feature to JetBlue.com that will allow passengers to rebook cancelled flights without having to call reservations.
Install up to 150 thin-client computers, with Internet connections, behind security at JFK and other hubs, to allow passengers to rebook flights at the airport using JetBlue.com.
Work with Navitaire to double the number of agents that can be accommodated on the Open Skies reservation system as soon as possible.
Enhance the new lost-bag tracking system so it can become a core application.
Implement new systems that will allow JetBlue to notify passengers by e-mail, phone or its Web site of cancelled or changed flights as soon as possible to prevent lines at airports.
Institute cross-training of staff on reservation, flight and crew scheduling applications.
In other words, bolster JetBlue's information infrastructure so that it can offer many of the features now being offered by its major airline competitors.