Lies, Damned Lies and Project Metrics (Part 1)

By Bruce F. Webster Print this article Print

Baseline introduces the column "Surviving Complexity" from author and IT expert Bruce F. Webster. In this introductory article, Webster takes a closer look at metrics vis-a-vis IT projects.

When Capers Jones published Assessment and Control of Software Risks (Yourdon Press, 1994), he identified the most serious software risk in IT projects as “Inaccurate Metrics,” and the second most serious software risk as “Inadequate Measurement”.  I remember being startled when I first read that back in 1995—they certainly weren’t what I would have chosen—and other authorities in the field criticized his choices. Yet, in the intervening years, I have moved closer and closer to Jones’ point of view.

Let me explain.

“Metric” is just a shorthand term for “that which is measured.” The idea is that by defining and measuring one or more metrics associated with some process, you can reach conclusions about that process. For example, in a manufacturing plant that makes widgets, a useful metric might be the number of faulty widgets that are detected and scrapped, while an even more useful metric might be the number of faulty widgets that aren’t detected at the plant but that are returned or complained about by unhappy customers.

The first metric tells you something about the quality of your manufacturing process, while the second tells you something about the effectiveness of your in-plant quality testing.

Likewise, in managing your IT project, you would like to have metrics that you can use to measure your project. Unfortunately, as Jones noted, most projects have problems with this. Let me explain why.

For a metric to be truly useful, it needs to have three characteristics. First, it needs to be informative or predictive; that is, it needs to give you relevant and useful information either as to what you’ve achieved so far or when you’re going to meet some milestone. This may seem obvious, but the single most popular metric in software engineering—number of source lines of code (SLOC)—has little informative value, particularly in light of practices such as refactoring.

What’s more, the number of source lines of code has no real predictive value, since it’s perfectly possible for the SLOC value to increase constantly yet for the project to get no closer to completion and, in some cases, to get farther away.

This article was originally published on 2008-06-05
Webster is Principal and Founder at at Bruce F. Webster & Associates LLC. He works with organizations to help them evaluate troubled or failed information technology (IT) projects, or to assess IT systems and products for possible investment/acquisition. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan.
eWeek eWeek

Have the latest technology news and resources emailed to you everyday.