ASQ - Software Division

Measuring and Managing In-process Software Quality

Abstract: In this paper we described the effort/outcome model for in-process metrics and quality management. The model goes beyond the traditional way of interpreting metrics and assessing quality when a software development project is underway. We illustrated the use of the model for the testing phase as well as for the front end of the development process with regard to design and code inspections. Improvement paths were discussed and possible effort and outcome indicators that can be used along the phases of software development life cycle were provided. The effort/outcome model and the metrics thus established, used, and interpreted, are clearly an effective way for quality management.

Keywords: Software - In-Process - Testing - Effort - Outcome - Kan - Defect - Measure - Improvement

Already a member? Access this Content

You will need Adobe Reader to view this PDF document.
Download the free Reader from Adobe

Brenda asks an interesting question about industry acceptable quality levels for software. There are none in general, but in many industries there is an implicit zero defects requirement, e.g., healthcare. The reality, of course, is that in most industries the reality is a defect rate of about 1-5 defects / KSLOC reported by the customer (out of about 40-50 defects / KSLOC injected in development). So the 10% suggested by Brenda isn't far off -- most software projects find 90-98% of the defects injected, with probably the majority around 98%. High-reliability software, e.g., healthcare, aerospace, can run significantly better, with some projects reporting near the ideal zero defects level, but that requires a lot of disciplined work. I have heard of one instance of an organization reporting, with millions of SLOC developed per year, very close to Six Sigma levels of quality.

Rather than industry standards, what we see more often are customer requirements for specific quality levels. Usually in high-reliability environments like aerospace where defects can have life-critical consequences and the only right answer is really zero. There are standards, like DO-178B, that require software safety analysis with multiple levels of rigor.
--Mark Paulk, 05-04-2012

Hello, are there any software industry standards on the acceptable rate of defects when software products go to production? For example, no more than 10% defects reported by customer.
--Brenda, 03-19-2012

--Jane, 05-21-2010

--Jane, 05-21-2010
  • Print this page
  • Save this page

Average Rating


Out of 0 Ratings
Rate this item

View comments
Add comments
Comments FAQ

ASQ News


Follow the
Software Division

Twitter  LinkedIn