Lies, Damned Lies and Project Metrics (Part 1) - Laws of Metrics

Second, the metric needs to be objective; that is, the metric’s value shouldn’t depend on who’s doing the measuring. Again, that seems obvious, yet the second most popular metric in software engineering is probably the “walking around” metric: You walk around to each developer and ask her or him, “How close to completion is [the module, class, subsystem, etc.] you’re working on?” And when he or she answers, “Oh, about 70 percent done,” you ask, “When do you think you’ll be finished, then?” to which he or she answers, “Oh, in about two weeks.”

This leads to two key laws of metrics:

        Weinberg’s Law of Metrics: “That which gets measured gets fudged.”

        The Metric Law of 90s: “The first 90 percent of a development project                 takes 90 percent of the schedule. The remaining 10 percent of the project             takes the other 90 percent of the schedule.”

Weinberg’s Law simply notes that if you’re asking me to pull a metric out of my, ah, head, I am most likely going to give you something that makes me look good—or at least lets me avoid any blame. The Law of 90s reflects the tendency in software projects to do all the easy parts first, which leads to a false sense of progress and completion, and thus inflated values for self-reported metrics.

Third, if possible, the metric should be automated; that is, you should be able to calculate the metric with the click of a mouse or the press of a key. This is important for several critical reasons. First, it goes a long way toward establishing the metric’s objectivity, since the value returned won’t—or at least shouldn’t—care who clicks the mouse/presses the key. Next, it makes collecting the metric painless and undemanding of human effort. This is important because of a third law of metrics:

        The Metric Law of Least Resistance: “The more human effort required to             calculate a metric, the less often (and less accurately) it will be calculated,             until it is abandoned or ignored altogether.”

Finally, the time spent by your IT engineers collecting and calculating that metric is time that they are not spending doing their actual jobs.

So, for your IT project, you want metrics that are informative and (if possible) predictive, objective and automated. In other words, you want to press a button and get a report that gives you some degree of information about how much progress has been made and when the project is likely to be completed.

Next week, I’ll discuss why this is so hard, but I’ll also talk about some possible solutions. Until then, I’ll see you on the bitstream.

Bruce F. Webster is an international IT consultant. You can reach him at bwebster@bfwa.com or via his websites at brucefwebster.com and bfwa.com.

© 2008 Bruce F. Webster