When Capers Jones published Assessment and Control of Software Risks (Yourdon Press, 1994), he identified the most serious software risk in IT projects as “Inaccurate Metrics,” and the second most serious software risk as “Inadequate Measurement”. I remember being startled when I first read that back in 1995—they certainly weren’t what I would have chosen—and other authorities in the field criticized his choices. Yet, in the intervening years, I have moved closer and closer to Jones’ point of view.
Let me explain.
“Metric” is just a shorthand term for “that which is measured.” The idea is that by defining and measuring one or more metrics associated with some process, you can reach conclusions about that process. For example, in a manufacturing plant that makes widgets, a useful metric might be the number of faulty widgets that are detected and scrapped, while an even more useful metric might be the number of faulty widgets that aren’t detected at the plant but that are returned or complained about by unhappy customers.
The first metric tells you something about the quality of your manufacturing process, while the second tells you something about the effectiveness of your in-plant quality testing.
Likewise, in managing your IT project, you would like to have metrics that you can use to measure your project. Unfortunately, as Jones noted, most projects have problems with this. Let me explain why.
For a metric to be truly useful, it needs to have three characteristics. First, it needs to be informative or predictive; that is, it needs to give you relevant and useful information either as to what you’ve achieved so far or when you’re going to meet some milestone. This may seem obvious, but the single most popular metric in software engineering—number of source lines of code (SLOC)—has little informative value, particularly in light of practices such as refactoring.
What’s more, the number of source lines of code has no real predictive value, since it’s perfectly possible for the SLOC value to increase constantly yet for the project to get no closer to completion and, in some cases, to get farther away.
