For organizations looking to implement ITIL and measure performance, Gallagher has some advice: "Don't promise improvement until you have a real good historical baseline."

Case in point: Before June 2004, Pershing had no baseline metrics for measuring severity-one incidents. When it started to develop processes for managing incidents and problems, the definition of a severity-one incident changed, and more incidents were reported.

Why? The company saw the value of reporting and tracking outages, Gallagher says. Incidents such as missing a service level agreement during the overnight batch cycle—not originally included—were added.

By December 2004—in time for the 2005 calendar year—Pershing locked down its definition of severe service outages. "We wanted a 12-month period of saying, 'This is how the organization performs without any changes to the criteria.' That was our baseline," Gallagher explains.

Once Pershing collected and analyzed metrics, the technology organization could create scorecards, another key tenet of ITIL, to show the performance of technology services—and, in effect, the performance of managers responsible for those particular services, or service owners, according to CIO Kumar.

One surprise: When Pershing created a catalog listing all technology services, Kumar says he and his team discovered that in some cases, there were multiple owners of a service. And in other cases, there were no owners. Plus, a service owner might not necessarily know who the customer of his service was. To improve accountability, Pershing has identified one service owner for each service, in a process that's still evolving throughout the Bank of New York, according to Kumar.

The CIO provided this example: Let's say there's an incident with a printing service that could be due to any of several reasons—either an application, network, firewall or server failing to work properly. Previously, the tech team would view the problem as either a hardware or software problem. Someone using the service, though, would view it as a printer problem, which is how the technology team now approaches its job.

Publishing a monthly scorecard—and identifying services and service owners—increased the transparency of technology operations and triggered an immediate improvement in operations, Kumar says.

"We didn't have to yell at people," he says. "We didn't have to offer any incentives. It just goes to show that when everyone comes to work, they love to do the best they can. But lack of information doesn't give them the knowl edge to see performance."