3.4 PER MILLION
Perusing Process Performance Metrics
Selecting the right measures for managing processes
by T.M. Kubiak
Often, I have students ask for clarification about the subtopic level in the 2007 Six Sigma Black Belt body of knowledge (BoK) that deals with process performance metrics. Perhaps the students are confused because of the sheer number of metrics or maybe because of some subtle differences among the metrics.
I hope this article can resolve a lot of the confusion surrounding the various process performance metrics. Based on my review of the literature,
I’m offering my interpretation of these metrics. I suspect many differing opinions will continue to exist, however, and there may be a flurry of letters to the editor after this appears.
Before I attempt to clear things up, it’s worthwhile to revisit each of the process performance metrics identified in the BoK, explore the relationships among the metrics and look at examples of each. The performance metrics include:
- Percentage defective.
- Defects per unit (DPU).
- Defects per million opportunities (DPMO).
- Parts per million (PPM).
- Rolled throughput yield (RTY).
- Process sigma.
Before examining each metric, however, it’s important to define "defect" accurately to provide a foundation for interpreting these metrics in a meaningful way.
Building the foundation
To use the defect-based metric effectively, it is important to answer:
- What is a defect?
- How can a defect occur?
What is a defect? I have always lived by the mantra that "anything not done correctly the first time is a defect." Of course, this means understanding what it takes to "do it right the first time." Simply put, a defect occurs during any process (for example, assembly, manufacturing, chemical or paperwork) when the outcome of the process is not the expected outcome. Of course, expected outcome means the conditions of a defect are specified in advance.
It’s important to hold true to these definitions. For example, if a unit was declared defective because it met the criteria, but a material review board later found it to be usable, the unit’s classification as defective should remain. Some organizations are reluctant to embrace this position because it adversely affects their quality numbers. Because a unit is usable, some organizations conclude it must not be defective. A defect may or may not affect usability.
For example, a new car may contain surface imperfections. By definition, the car is defective because it contains one or more defects, but it is still usable. Reclassifying a defective unit as a nondefective does nothing to help resolve the underlying cause of the defect.
How does a defect occur? To address this question, many organizations have compiled a list of defect families and defect types within families. Such a list should be as complete as possible in identifying all possible defect types. Also, each defect type should be independent and mutually exclusive of others. This allows you to recognize the occurrence of multiple defects on any given unit.
Furthermore, avoid the temptation to exclude known defect types because they happen infrequently. In other words, if a defect family or type is known to occur, include it on your list. In addition, it is useful to have a defect family or type deemed "other" because there may be a lack of foresight or wisdom to define everything in advance.
As you develop your list of defect types, it is often useful to define them in pairs (for example, too high and too low, or too long and too short), particularly when you are examining physical, mechanical or electrical characteristics. You might argue that a type such as "too long" or "too short" should simply be defined as one defect (for example, the wrong length). This is a viable argument and worth considering. I would suggest, however, looking beyond the defect to the action the defect creates.
For example, if the "too long" defect results in a unit requiring further trimming and rework, and "too short" requires the unit be scrapped, the consequences of the defect occurrence are different. Different consequences may require identifying and tracking different defect types. Classifying the defect occurrence as two different defect types allows for future root cause analysis.
Defining the metrics
When selecting meaningful metrics, consider the audience and how the metrics will drive action. In the following example, the PPM metric might be more understandable to an organization’s management that compares processes at a high level. A quality engineer who has oversight responsibility for the process, however, may consider the DPU metric to be more actionable at the specific process level.
As you read through each of the process performance metrics below, consider how they may apply within your own organization.
The percentage defective is simply defined by the following equation:
Of course, a defective unit is any unit containing one or more defects. Note that the ratio,
is known as the fraction defective.
Consider a process in which the output is normally distributed with a mean of 0 and a standard deviation of 1. Specifications are set at +/- 3. The fraction defective for the process is shown by the tail areas in Figure 1. The total fraction defective is the sum of the tail areas, or 0.0027. Therefore, the percentage defective is 0.27%.
The DPU metric is a measure of capability for discrete (attribute) data defined by the following:
For example, a process produces 40,000 pencils. Three types of defects can occur. The number of occurrences of each defect type is:
Blurred printing: 36
Too long: 118
Rolled ends: 11
Total number of defects: 165
A straightforward application of the DPU formula provides this:
The DPMO metric is a measure of capability for discrete (attribute) data found by:
The DPMO metric is important because it allows you to compare different types of product. Developing a meaningful DPMO metric scheme across multiple product lines, however, can be very time consuming because it is necessary to accurately determine the number of ways (or opportunities) a defect can occur per unit or part. This can be an enormous task, particularly when dealing with highly complex products and subassemblies, or even paperwork.
Continuing with the pencil example, let’s calculate the number of opportunities. First, determine the number of ways each defect can occur on each item. For this product, blurred printing occurs in only one way (the pencil slips in the fixture), so there are 40,000 opportunities for this defect to occur.
There are three independent places where dimensions are checked, so there are (3) (40,000) = 120,000 opportunities for this dimensional defect.
Rolled ends can occur at the top and the bottom of the pencil, so there are (2) (40,000) = 80,000 opportunities for this defect to occur. Thus, the total number of opportunities for defects is:
40,000 + 120,000 + 80,000 = 240,000.
Likewise, the total number of opportunities per unit is:
1 + 3 + 2 = 6
Applying the DPMO formula, you can readily determine the DPMO metric:
In a typical quality setting, the PPM metric usually indicates the number of times a defective part will occur in 1 million parts produced. By contrast, the DPMO metric reflects the number of defects occurring in 1 million opportunities. It is important to note that some authors say the PPM and DPMO metrics are identical. If we follow the definitions above, however, this would only be true when the number of opportunities for a defect per unit or part is 1.
Perhaps additional confusion can surround the PPM metric because of a laxness in the terminology applied. In the Six Sigma context, PPM is also referred to as the PPM defect rate. Similarly, 3.4 PPM is often stated as 3.4 defects per million parts. In both examples, however, when we say defects, we are really referring to defectives.
PPM is also used to refer to contaminants. For example, suppose 0.23 grams of insect parts are found in 25 kilograms of product.
Finally, in the more traditional scientific context, PPM may simply refer to the various ratios of components in a mixture. For example, the oxygen component of air is approximately 209,000 PPM. In this case, the idea of "defective" isn’t even a consideration.
Table 1 illustrates the links among multiple metrics, including PPM, sigma level, percentage in specification and percentage defective. The familiar 3.4 PPM corresponds to a 6-sigma level of quality, assuming a 1.5 shift of the mean. Sigma level of a process and the 1.5 shift of the mean will be addressed later.
The RTY metric represents the percentage of units of product passing defect free through an entire process. It is determined by the multiplying first-pass yields (FPY) from each subprocess of the total process as follows:
Note that n= number of subprocesses, and FPYi = first-pass yield of the ith subprocess.
Similarly, the FPY represents the percentage of units that completes a subprocess and meets quality guidelines without being scrapped, rerun, retested, returned or diverted to an offline repair area. The FPY is calculated as:
Note the FPY and RTY values are often expressed simply as the fractions or probabilities.
The concept of the RTY is best illustrated by the example given in Figure 2, which depicts an overall process comprised of four subprocesses. Suppose the FPY of each subprocess is 0.95. Then, the RTY is easily computed as:
Although individual subprocess yields are relatively high, the total process yield has dropped significantly. A significant advantage of using the RTY metric is that it provides a more complete view of the process. Subprocess yields that run high aren’t likely to garner the attention necessary to drive improvement. Often, it is only when the total process yield becomes visible does real action occur.
When there’s talk of the process sigma of a process, you’ll often hear it described as a 3-sigma or 4-sigma process or something similar. Sometimes you’ll hear it described as the sigma level of a process. What does this mean and how do you interpret it in the context of Six Sigma?
Assume the output of a process is operating as a standard normal distribution with a mean of 0 and standard deviation of 1, with an upper specification limit (USL) and lower specification limit (LSL) set at +/- 3, respectively. This is depicted by the blue curve in Figure 3. From basic statistics, you know that:
P (Z ≥ 3 = USL) = 0.00135 (the area to the right of the USL and below the blue curve in Figure 3).
P (Z ≤ 3 = LSL) = 0.00135 (the area to the left of the LSL and below the blue curve in Figure 3).
This gives a total fraction defective 0.0027 or percentage defective of 0.27% as we determined previously.
The underlying philosophy of Six Sigma, however, assumes a 1.5-sigma shift of the mean either to the right or left over the long term. If you assume the shift is to the right as shown in Figure 3, the process distribution is normal with a mean of 1.5 and a standard deviation of 1. Applying basic statistics again, you know that:
P (Z ≥ 3 = USL) = 0.0668072 (the area to the right of the USL and below the red curve in Figure 3).
P (Z ≤ 3 = LSL) = 0.0000034 (the area to the left of the LSL and below the red curve in Figure 3).
Note that the area is small and therefore difficult to depict graphically.
This results in a total fraction defective of 0.0668106, a percentage defective of 6.68106% and a PPM level of 66,811. From Table 1, we can see these values equate to a 3-sigma level.
If you extend the same approach as above, you can quickly generate the values shown in Table 1.
A quick review of Table 1, along with understanding the 1.5-sigma shift, explains why Six Sigma uses 3.4 PPM for a 6 sigma process, and not 2 PPB.
In addition to the use of Table 1, the sigma level associated with the 1.5-sigma shift can be approximated based on the PPM metric using the following formula:
The above equation very closely approximates the actual sigma value when the PPM is below 309,000, or when the sigma value is expected to exceed 2.
The subject of the 1.5-sigma shift is highly controversial. You may or may not accept its validity. The intent of this section was simply to explain how the shift relates to PPM and associated process sigma level.
Clearly, there are a lot of process performance metrics to consider. Though they take different forms, some of them are equivalent. Selecting the appropriate ones for your organization depends on your audience and how the metrics will be used to drive improvement actions.
Some metrics are more understandable than others, while some have more of an emotional impact. For example 66,811 PPM may be more startling to management than using a corresponding 3-sigma level. Regardless of which metrics you choose, each one must be based on a clear operational definition of a defect.
By the way, I’ve had numerous students ask me what happens to the defective products shown in Figure 2. I tell them they are packaged as complex derivatives and other high-quality securities and sold on the world financial markets. Some just sit there and wonder. Others think, "Good idea!"
Breyfogle, Forrest W. III, Implementing Six Sigma: Smarter Solutions Using Statistical Methods, second edition, John Wiley & Sons Inc., 2003.
Kubiak, T.M. and Donald W. Benbow, The Certified Six Sigma Black Belt Handbook, second edition, ASQ Quality Press, 2009.
Kubiak, T.M., Expert Answers, Quality Progress, June 2008, p. 12.
Schmidt, S.R. and R.G. Launsby, Understanding Industrial Designed Experiments, Air Academy Press, Colorado Springs, CO, 1997.
T.M. Kubiak is an author and consultant in Weddington, NC. He is a co-author of The Certified Six Sigma Black Belt Handbook. Kubiak, a senior member of ASQ, serves on many ASQ boards and is a past chair of ASQ’s Publication Management Board.