Bruce F. Webster: Instrumentation

By Bruce F. Webster  |  Posted 2008-06-18 Email Print this article Print
 
 
 
 
 
 
 

Instead of deciding ahead of time which metrics are likely relevant, you first want simply to gather as many metrics, or characteristics, of your IT project as you can, then process them

This is where instrumentation comes in. Instrumentation simply means setting up your IT project so you can track as much information as possible about its different aspects and characteristics, preferably in an objective, automated manner. One place to start is your SCM (software-configuration management) system, which should be tracking, not just source-code changes, but changes and versions for all project deliverables (requirements, specifications, models, diagrams, test plans and results, and so on).

This means that your instrumentation will likely need to be able to (programmatically) go into your SCM system and extract that information. With this access, you want to extract characteristics, including size, changes, attributes and authorship, from various files and deliverables.

Next, you want to extract information from your defect-management system (you do have one, don’t you?). You would gather information about the number of defects reported, closed to date, deferred, marked as duplicate, marked as enhancements and so on. You would track both total numbers and new reports for each unit of time (say, day or week), as well as the severity and priority of reported defects.

In conjunction with your SCM and defect-management systems, you also want to gather information about your change-control process, that is, the meetings you hold to prioritize defects and to approve or defer features. Another area to track is human information such as organizational structure, turnover, task assignments, background and qualifications of individuals, and meeting schedules. Again, as far as possible, this information-gathering should be objective and automated.

At this point, you’re probably throwing up your hands and saying, “Wait! Stop! That’s too much!”

My response to that is: We are willing to spend millions, tens of millions, even hundreds of millions of dollars on projects that fail, so why are we so reluctant to spend a tiny fraction of that to help those projects succeed? (The same short-sightedness applies to software quality assurance, but that’s a subject for another column.)

Beyond that, the advantage of gathering so much information is that it makes it very difficult to fudge that information. A developer or manager might be able to do that with one or two particular characteristics but not with all the characteristics you’re monitoring. What’s more, there’s a good chance that attempts to fudge a particular characteristic will show up due to inconsistencies with the other information being gathered.



<123>
 
 
 
 
Webster is Principal and Founder at at Bruce F. Webster & Associates LLC. He works with organizations to help them evaluate troubled or failed information technology (IT) projects, or to assess IT systems and products for possible investment/acquisition. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan.
 
 
 
 
 
 

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters



















 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Thanks for your registration, follow us on our social networks to keep up-to-date