Topic: Open-source bug hunt results  (Read 995 times)

0 Members and 1 Guest are viewing this topic.

Offline Nemesis

  • Captain Kayn
  • Global Moderator
  • Commodore
  • *
  • Posts: 13067
Open-source bug hunt results
« on: May 10, 2006, 10:30:58 am »
Link to first article

Quote
Coverity Inc. of San Francisco has released the results of a Homeland Security Department-funded bug hunt that ranged across 40 popular open-source programs. The company found less than one-half of one bug per thousand lines of code on average, and found even fewer defects in the most widely used code, such as the Linux kernel and the Apache Web server.

The results are the first deliverable of a $1.2 million, three-year grant DHS awarded to a team consisting of Coverity, Stanford University and Symantec Corp. of Cupertino, Calif. DHS wants to reinforce the quality of open-source programs supporting the U.S. infrastructure.

The agency is hoping developers will fix the defects highlighted by the team’s advanced bug-hunting techniques. Such defects can pose security vulnerabilities because they could be used by malicious programs to disrupt or gain control of a system.


Quote
Generally speaking, it is difficult to determine how well these open-source programs compare with their proprietary counterparts, Chelf said. Coverity has tested only a few commercial products, so direct comparisons cannot be made.


Link to 2nd article

Link is to a page with a table of the number/thousand lines in the various projects.

To make it clear this was an automated bug hunt and would not find logic errors such as connecting a button labeled "Save" to the format disk command.  These though are errors that can lead to many of the attacks that do happen.  They can be as simple as an opening bracket ( with no closing bracket ). 

I list a few of the higher profile projects below along with their bug/1000 line value.

apache-httpd 0.257
Firefox 0.275
FreeBSD 0.396
gcc 0.284
Gnome 0.078
KDE 0.039
Linux-2.6 0.206
NetBSD  0.001
OpenOffice.org 0.328
Perl 0.012
PHP 0.183
Samba 0.016
Thunderbird 0.183
Do unto others as Frey has done unto you.
Seti Team    Free Software
I believe truth and principle do matter. If you have to sacrifice them to get the results you want, then the results aren't worth it.
 FoaS_XC : "Take great pains to distinguish a criticism vs. an attack. A person reading a post should never be able to confuse the two."