The Probability Problem

By Regina Kwon  |  Posted 2001-12-10 Email Print this article Print
 
 
 
 
 
 
 

It's Monday morning, and the chief operating officer calls you in for advice.

It's Monday morning, and the chief operating officer calls you in for advice. The new sales-commission structure is too complicated for the current finance application, so several staffers have been calculating percentages and inputting payments by hand. The upgrade planned for the financial software will take care of this, but that rollout is at least eight months away.

Can your developers build a standalone application within six weeks, the latest date that would make the project worthwhile?

Numerous cognitive research studies have found that people are consistently overconfident in their judgment by about 20%—putting the likelihood you'll deliver that finance application within six weeks at about 70%. And if you're a manager, your estimate is probably more than twice as generous: In a study of 1,000 American and European managers, business school professors J. Edward Russo and Paul J. H. Schoemaker found that group to be correct only 30% to 60% of the time.

View the PDF -- Turn off pop-up blockers!

Most risk models rely on estimates made by subject-matter experts, says Douglas Hubbard, the inventor of Applied Information Economics, a methodology for calculating return on IT projects. Overconfidence can cost a company large sums, as well as incur bad blood between business and information technology. To correct for this, Hubbard starts each engagement with a workshop in which he "calibrates" participants to appropriate confidence levels.

A typical calibration workshop consists of a half-day of general-knowledge questions, in which participants provide answers or estimates along with their confidence that the answers they gave are correct.

"After a few sessions, people get a sense of what a 90% confidence level actually feels like," says Hubbard, who recently conducted workshops for research firm Giga Information Group and consulting company Booz Allen Hamilton. "They stop using 90% so much, but when they do, they're usually right."

Background Reading

For basics on calibration, read Russo and Schoemaker's Decision Traps. For more about AIE and calibration workshops, visit www.hubbardresearch.com.



 
 
 
 
As Statistics Editor of Baseline magazine, Regina creates interactive tools, worksheets and project guides for technology managers. Before joining Ziff Davis, she worked as a technical program manager for a database company, where her projects included data management applications in XML, Java, Visual Basic and ASP. Her other experience includes running the new media department at Christie's Inc. and writing and editing for Internet World and PC Magazine. Regina received a B.A. from Yale.
 
 
 
 
 
 

Submit a Comment

Loading Comments...

Manage your Newsletters: Login   Register My Newsletters