Don’t Become Awash in Useless Metrics

Every month, my predecessor distributed a 30-page report to each executive and manager about the information-technology infrastructure at Rohm and Haas. He fed everyone a slew of results on performance metrics such as how well the processors on our servers were being utilized. His goal: reach out to explain the technology infrastructure at our company, which makes specialty chemicals and produces Morton Salt.

When I became director of global infrastructure in 2001, three people asked me to take them off the report’s distribution list. The concept of communicating information-technology performance was good, but it wasn’t providing value to employees. Not only did I take them off the list, I stopped publishing the report.

I don’t like to do monthly reports. Instead, I like to work with my team to identify key metrics to determine whether or not systems are running well. My approach: create a snapshot, using data charted on a graphic, that shows trends over months.

Take a traditional metric such as the CPU or processor utilization rate. It’s meaningless because it fails to give a full picture of what’s going on with information systems.

If multiple processors don’t share the work of running an application very well, the processor utilization rate may show the server running at 30% capacity. In reality, some processors may be at capacity and others barely idling. As a result, the average rate will mask a slowdown in handling transactions or other problems.

While we still collect CPU utilization, we only look at it in a gross sense to see changes in historical trends.

Now, we focus on metrics that support business needs.

Consider this: Rohm and Haas made 40 acquisitions and divestitures in the late 1990s, ending up with 40 different enterprise resource planning systems. Some were basic general ledgers or other rudimentary setups. We consolidated them into a single instance of the SAP planning system. SAP now manages our ERP transactions including financial, manufacturing, customer orders, billing, vendor purchases, supply and demand planning.

After the initial rollout of SAP, we had a major outage. I decided we needed overall targets to measure and improve performance. I chose two: 1. Make sure the system is available 99.5% of the time; and 2. Process 95% of all transactions, such as a sales order, in under one second.

What’s the value of measuring the performance of transactions over time? In this instance, we noticed sluggishness before there was pain. Or, in other words, before a customer was in our face with a problem.

We found we were processing 93% of all transactions in July and August 2004 in under one second, instead of 95%. After exhausting options to tune up performance, we upgraded the database server, which improved response time. When response time dropped off three months later (when another business went live), we upgraded the application servers and storage. That fix was short-lived.

When the performance dropped off again (more businesses went live), we discovered we were sending so many transactions over a fiber link that connected the production servers at our Philadelphia headquarters to a backup site 20 miles away, that the pipe was full. Once we replaced the network, the performance improved to better than 96%.

It’s important to use metrics to piece together problems, identify action plans for tuning up equipment or making hardware upgrades, and then track that action to see if it resolves a particular problem. If you don’t monitor the performance of information-technology operations, your tech staff will jump from fire to fire extinguishing immediate problems instead of working on continuous improvements.

Unless you can prove that your information technology is good and cost effective, you may end up giving your business manager no other choice but to put out the fire altogether—by outsourcing your job and your company’s technology operations.

Randall W. Carter is I.T. director of infrastructure operations at Rohm and Haas. He can be reached at [email protected].

Written with Anna Maria Virzi