Coverity, in collaboration with Stanford University and under contract from the Department of Homeland Security, has just released their Open Source Report 2008. Their environmental scan of major open source projects found that the number of defects in open source code is dramatically dropping! More detail is available on ArsTechnica.
Coverity’s Web site mentions Stanford several times, and the $1.24 million contract was split between them; however, I couldn’t find any information about which department or researcher(s) worked on the project either on coverity, Stanford web space or the report itself (which doesn’t mention Stanford at all!). Do any oslblog readers know who worked on this most interesting project?
In 2006, Coverity’s scan detected an average of 0.30 defects per 1,000 lines of code, or, put differently, one code defects per every 3,333 lines. The lower boundary, in this case, was 0.02 (one defect per 50,000 lines) and the upper boundary was 1.22 defects per thousand lines of code.
Two years later, the average defect density has fallen to 0.25, or one error per 4,000 lines of code. The upper boundary remains unchanged at 1.22, but the lower boundary has shrunk to 0, implying that repeated scanning has eliminated the errors from at least one program—at least all the errors that Coverity’s 2006 static analysis program was able to detect.
A 16 percent reduction in defect density over two years is a notable gain, and Coverity singled out certain participating projects as having an exceptionally low defect density.
