Putting Your Finger on the Pulse


Software that measures application performance doesn’t live up to the most spectacular vendor promises—and it’s harder to set up than most people realize. But such tools can be the difference between a system slowdown and a show-stopping disaster.

In Nielsen Media Research’s data center in Oldsmar, Fla., near Tampa, about 700 servers hum away in air-conditioned comfort. Marty LeFebvre, the television-ratings company’s vice president of technology strategy, is most concerned with 200 of those, which host proprietary data-gathering and analysis applications at the very heart of Nielsen’s service.

How to tell when an application is on the brink of collapse? Five years ago, LeFebvre and his staff had no consistent, centralized way to find out. To keep a handle on the performance of the servers and the applications running on them, Nielsen decided to roll out BMC’s Patrol system-management software.

The goal was simple. But reaching it certainly wasn’t, LeFebvre says: “There are dozens of moving parts in being able to define an application.” One problem was information overload. The servers reported on each and every little thing that was wrong with them through Patrol. Every day, Nielsen’s systems generated 232,000 alerts, which included any event outside the bounds of normally anticipated behavior.

Nielsen took about a year to get fully comfortable using Patrol as part of daily operations. LeFebvre and his staff have since tuned the software to filter the alerts down to between 200 and 300 events per day, which are sent to administrators to act on. While LeFebvre believes the system lets the company manage its information resources more proactively, he says he wishes he had initially committed more staff and money to get the project off the ground faster. “In hindsight, I would have much more realistic expectations about where it would take us, and what resources we needed,” he says.

In the world of measuring application performance, hype and inflated expectations have outpaced reality. The latest vendor chatter is about “business-service management,” a buzzword describing products that are supposed to allow customers to manage information-technology resources according to business processes, by automatically identifying the infrastructure problems directly affecting them. But customers say nobody’s really delivering the complete package yet.

Steve Hernandes, director of enterprise management for financial-transaction services company First Data, says any management product on the market requires extensive effort to figure out what makes an application tick. “I don’t care whose tool you’re using. There’s nothing that can automatically discover which pieces of the application are associated with which pieces of the infrastructure,” he says. “Today, you have to put work into these tools.”

The sweat can pay off. First Data uses Computer Associates’ Unicenter, which it deployed in 2000, to closely track 5,000 servers running a mix of AIX, Solaris, AS/400 and Windows operating systems. Hernandes says the CA software has helped cut his staff’s response time to critical outages from about 30 minutes to a few seconds.

“In the old days, we had to wait for the customer to call us,” he admits. “Today it’s all automated. When we see a server fail, we page the administrator responsible for it.”

Because the products are evolving, some technology managers are reticent to dive into application-performance monitoring. Burlington Coat Factory has been using Mercury’s testing tools since 1998 as part of its software-development process. Bruce Woods, the retailer’s manager of software quality assurance, is holding off from buying into any particular vendor’s approach for application management even though his group has a clear need for the technology. “I’m not willing to pull the trigger until we get a best-of-breed product,” he says. “I want to get the feeling there’s an 800-pound vendor.”

The closest thing to gorillas here are the traditional enterprise systems management vendors: BMC, CA, Hewlett-Packard and IBM’s Tivoli unit. Mercury, whose roots are in software-testing tools for developers, is a relative newcomer to the operations field. Historically, these players have focused on monitoring individual infrastructure elements instead of how to correlate them and show a complete picture of the health of an application, which can depend on any number of servers, operating systems, network devices, storage arrays and application services.

“The industry has done a good job of being able to measure whether a server isn’t performing well,” says Meta Group analyst Glenn O’Donnell. “But then the first question after that is, ‘What went wrong?'”

The vendors are trying to help customers drill down on those questions better. CA’s Unicenter lets customers manually set up “business process views” to link applications with specific business functions. By the end of 2004, the company expects to augment the feature to build such views automatically with the Sonar technology it bought last year from Raytheon.

Last month, HP launched OpenView Business Process Insight, which calculates the performance of information-technology resources based on financial information. For example, HP says, the software could report that 15 orders are stuck in credit check, stopping up $200,000 in revenue.

Mercury, meanwhile, this May acquired Appilog, a 40-person startup whose software creates self-updating “maps” that show relationships among network, server and application elements. “It’s like a dynamic MRI of your environment instead of an X-ray,” says Chris Lochhead, the company’s chief marketing officer.

In practice, though, how successfully a company can manage applications in relation to the systems and networks that underpin them resides in its own DNA. “I can’t tell you how many organizations have server groups and network groups that don’t talk to each other,” says Dave Wagner, BMC’s director of enterprise assurance solutions.

And those on the receiving end of vendor promises remain doubtful that automated application-management technologies will come off anytime soon. As First Data’s Hernandes says of CA’s Unicenter plans: “I’ve got to see it before I believe it.”

Group Dynamics: Box Watchers

Category: Application-performance management software

What It Is: Software that tracks availability and performance of business applications and the systems running them, alerting operations staff of deviations from baseline performance.

Key Players*: BMC, Computer Associates, Compuware, Hewlett-Packard, IBM, Mercury Interactive, Quest Software, Veritas Software

Market Size: $2.58 billion in worldwide revenue for performance management software, 2003 (IDC)

What’s Happening: Vendors are improving their products’ ability to correlate system-level information with business processes.

Expertise Online: tek-tips.com hosts several discussion forums on enterprise management tools.