Weeding Out Defects in Software Development

To reduce costs, a lot of companies are sending software development work offshore. We have not.

Instead, we at the Vanguard Group asked: What if we take 65% of the capacity used to test and rework errors in software development, and use that to build new software?

Our ammunition came from a CIO survey that found about 30% of the work on projects goes toward testing and reworking software, which does not add any new functionality to the underlying code.

If we could bring that percentage down to 10%, we figured we could spend more time developing applications that benefit our business and customers—such as a new Web site serving the needs of financial advisers.

The Vanguard Group constantly invests in its information systems because we are, in effect, a virtual company. We’re the nation’s second-largest mutual fund firm with $850 billion in assets and 18 million shareholder accounts. Our clients use our services through Vanguard.com or by phone. Technology helps to differentiate us from our rivals, and more importantly, allows us to serve our clients better.

With more than 1,800 software developers in Valley Forge, Pa., and Charlotte, N.C., about 70% of our resources go toward building custom software, such as our Web site or trading systems, and only 30% toward installing packages.

At about the time we looked to improve software quality, our company kicked off a campaign called Unmatchable Excellence. It takes the best components of Six Sigma, Michael Hammer’s “process excellence” and other methodologies for process improvements.

Under Six Sigma or process excellence, the first order of business is to standardize your processes. Three major divisions build software at Vanguard: one for retail clients, another for 401(k) defined-contribution clients and one for corporate customers. Initially, many of us thought we had a single process for our software development. When each division wrote down the steps it took, we discovered that software was being written three or more ways. To standardize that work, we created cross-divisional teams for every process during software development such as requirements, environments and testing. Each team mapped out existing procedures and came up with an ideal process to be applied to its particular area of work.

The next challenge: defining a defect, and then coming up with metrics on how to track and then eliminate those defects. In production, for example, we defined a defect as a software change request. That is, if something is broken or wasn’t built properly, that’s considered a defect.

Once we began tracking these defects back to our development process, it became clear where we could improve. For instance, we discovered our unit testing, or testing of individual software modules, was lacking and could be improved with a simple tool.

Next came our “BHAG,” or Big, Hairy, Aggressive Goal for software quality. In the past, meeting deadlines and budgets was usually cited as our foremost goal. We had to change the mind-set and say, “Quality is paramount.” In effect, our BHAG is to reduce defects by 50%.

Continued attention to the BHAG and a competitive spirit changed our culture to quality. The result was phenomenal: Defects were cut by up to 65% on some projects.

Over the past three years, our resources dedicated to testing and rework have dropped by about 30%, or the equivalent of 180 full-time developers. As a result, we have been able to pursue projects that would have likely remained on the back burner longer.

People will tell you that Six Sigma is intended for production factories where processes are repeated, and not for software development. While we don’t create the same software over and over again, we do use the same processes. By standardizing our processes for writing software code, we have demonstrated that we can improve the quality of the software we produce, and raise our capacity.

—Written With Anna Maria Virzi