December 2003
Volume 6 • Number 1

Contents

VERIFICATION AND VALIDATION

Identifying Code-Inspection Improvements Using Statistical Black Belt Techniques

Jeff S. Holmes

INTRODUCTION

Peer reviews or inspections consist of techniques that utilize people to visually review software code. These techniques have proved very effective in removing defects from software. The removal of defects in the coding phase is significant because identifying defects and removing them prior to test is significantly cheaper. Peer reviews are only one tool that is used in preventing defects from escaping to the customer, but it is a very powerful tool that may be underutilized.

Industry sources identify that the expected numbers of defects in software are between 10 defects per thousand lines of source code (KLOC) and 100 defects/KLOC (Humphrey, 1995). Even using the conservative end of the range, a software team can expect to have 1000 defects for every 100,000 lines of code (LOC) written. Most of these defects must be removed before the software enters system test.

From January 2002 through April 2003, the firmware teams from Motorola Base Transceiver System Center of Excellence (BTS COE) in Fort Worth, Texas, performed 207 code inspections. These 207 code inspections were performed on 40,132 LOC, took more than 1428 staff hours, and removed 1345 defects. These inspection results indicate that the Fort Worth software defect density is at least 33.51 defects/KLOC. For every 100,000 LOC written, the Fort Worth software could have 3351 defects injected. These numbers do not include the defects caught by subsystem and system test, so it is a safe assumption to imply that the defect density is more than 33.51 defects/KLOC.

If the inspection effectiveness can be improved by 25 percent for the next 40,000 LOC developed by the Fort Worth teams, approximately 336 additional defects can be prevented from reaching the test teams and possibly the customer. There are various sources sighting the additional cost of removing defects in test versus code (McConnell, 1993). These sources claim that it can cost from four to hundreds of times more to remove defects in the test cycle rather than in code. Hypothetically, if each defect costs $100 to remove in code, the test cost could be $400 or greater per defect. Improving the inspection effectiveness by 25 percent could result in savings over $100,000 on the next 40,000 LOC developed by the Fort Worth team.

The need for effective defect removal has never been greater. Staff reductions and tight deadlines require software development teams to become more efficient in all aspects of their work. The 1400-plus hours of staff time spent in inspections represents a significant investment by the software development teams. This statistical analysis strives to examine the data on the inspections performed over this time in order to increase the effectiveness of code inspections.

This project seeks to answer the following questions about BTS COE Fort Worth code inspections.

• What are the significant factors in increasing the effectiveness of code inspections?
• How can the Fort Worth software teams increase their defect removal capabilities using inspections?

Using the extensive data set that exists on Fort Worth inspections, and applying the Six Sigma Black Belt techniques of statistical process control, advanced linear regression, and design of experiments, this document identifies the significant factors affecting an inspection’s effectiveness. Using these factors, the article reveals where the inspection process can be improved and the expected improvements in terms of increased defect detection and dollars saved.

Return to top