BACK TO BASICS
Zero Defect Sampling
by Tony Gojanovic
One perk of being a quality professional is running across a statistical rule of thumb that has application to the quality sciences. One such gem is the rule of threes—a paper and pencil technique that provides a simple way to add up risk around defect-free claims.
The rule of threes is the upper bound of a 95% statistical confidence interval for how poor quality can be consistent with finding zero defects in a random sample. The formula for the upper bound of the interval is
p = 3/ n
in which a random sample of size n is used to determine the upper limit risk, or p. The lower bound of the confidence interval will always be zero. In other words, the 95% confidence interval would be written as 0, 3/n. What the confidence interval generalizes is a range of values likely to contain the true defect rate given zero observed defects in a random sample of size n.
Rule of 3s
For the mathematically curious, the rule of threes can be derived from the binomial model. Briefly sketched, if given a sequence of n independent trials—with a pass or fail outcome on each trial—the test statistic for the defect proportion, p, will follow a binomial distribution with parameters n and p. The probability of seeing zero failures in n trials is computed with the binomial distribution as (1–p)n. Setting (1–p)n = 0.05 (the 5% risk level), and using natural logarithms to solve the equation, we have n ln(1–p) –3. For small p, ln(1–p) – p so – np = –3 or p = 3/n. The rule of threes works best for sample sizes of 20 or more. A similar derivation can be obtained with the Poisson model. —T.G.
The 95% simply refers to a 0.95 probability that the confidence interval constructed from the initial sample captures the true defect rate. The value of the confidence interval in quality applications is to temper overly optimistic expectations associated with finding zero defects in a sample with a statement of potential defect values. In the instance of actually having the worst case risk, it can be shown that the 95% implies the probability of finding one or more defects in the sample.
The formula is only valid for randomly selected samples, or if the defects are randomly distributed through the population under investigation. The following examples show the wide applicability of this formula—each case assumes inspection error is negligible, which might not always be the case.
Table 1 lists some other approximations for different confidence intervals that are useful.
Problem: What is the risk statement for 100 randomly selected electronic components from an assembly line that are tested for a specific defect if zero defects are observed?
Solution: Using n = 100 and the rule of threes, there is an upper level risk of 3/100, or 0.03, with 95% confidence. Given the risk level selected, the actual failure level could be between 0% and 3%, and still be consistent with zero observed defects in a sample of 100. If the lot actually contains the worst case rate of 3% defects, the likelihood of finding one or more defects in the sample is 95%.
Problem: A beer bottle manufacturer informs the quality
manager of a large brewery that the bottling process is
generating a certain type of defect, and he thinks the defect
proportion might be 1 in 5,000 bottles (0.02%) or higher. You
have a large amount of suspect pallets on hold, and you need to
find out if the hold has defective bottles in it. What sample
size do you need to ensure that you will find one or more defects
with 95% confidence?
Solution: Working backward using the rule of threes, the following facts are known: p = 0.0002 (the same as 0.02%) and n is not yet known. The rule is p = 3/n, which implies n = 3/ p. Solving for n, there is n = 3/p = 3/(0.0002) = 15,000 bottles.
At least 15,000 bottles would need to be randomly inspected to have 95% confidence of capturing one or more defective bottles.
There are other situations in which the question might address a defect level consistent with one defect found, and rules can also be derived for those situations. But more commonly encountered in practice, and often more critical, is the situation when zero defects are found in a sample and the free claim is to be evaluated in terms of defining the potential risk.
- Hanley, James A. and Abby Lippman-Hand, “If Nothing Goes Wrong, Is Everything All Right?” The Journal of the American Medical Assn., April 1983, Vol. 249, No. 13, pp. 1,743-1,745.
- Jovanovic, B.D. and P.S. Levy, “A Look at the Rule of Three,” The American Statistician, May 1997, Vol. 51, No. 2, pp. 137-139.
- Louis, Thomas A., “Confidence Intervals for a Binomial Parameter After Observing No Successes,” The American Statistician, August 1981, Vol. 35, No. 3, p. 154.
- Van Belle, Gerald, Statistical Rules of Thumb, John Wiley & Sons, New York, 2002.
TONY GOJANOVIC is a statistician at Coors Brewing Co., Golden, CO. He earned his master’s degree in mathematics and statistics from the University of Colorado in Denver. He is a member of ASQ.