MEASURE FOR MEASURE
Improved Gage R&R Measurement Studies
by Donald S. Ermer
Many manufacturers are using tools like statistical process control (SPC) and design of experiments (DoE) to monitor and improve product quality and process productivity. However, if the data collected are not accurate and precise, they do not represent the true characteristics of the part or product being measured, even if organizations are using the quality improvement tools correctly.
Therefore, it is very important to have a valid quality measurement study beforehand to ensure the part or product data collected are accurate and precise and the power of SPC and DoE are fully realized. Accuracy—in other words, no bias—is the function of calibration and is performed before a correct measurement study of the precisions of the gage and its operators.
In this two-part column, I will review the gage repeatability and reproducibility (R&R) study in the Automotive Industry Action Group (AIAG) manual1 for its ability to determine the true capability of different parts of a measurement system. I’ll use a geometrical approach to describe the components of the total measurement variance. This shows why the standard deviations or measurement errors of the equipment, appraiser and product in the AIAG method are not additive and cannot be compared directly in a ratio.
In part two, which will appear in the May issue of QP, I will provide a worksheet for correctly executing a measurement process capability study that combines the advantages of my improved measurement study.
The Gage R&R Study In the AIAG Manual
The gage R&R study in the AIAG manual uses a data collection system that is well structured and very helpful in collecting the proper data. The data are then used to calculate the standard measurement errors, or standard deviations, of the equipment, appraiser and product. The total measurement error, or standard deviation, is then obtained by taking the square root of the sum of the squares of the equipment, appraiser and product standard deviations.
Next, the measurement ratios are calculated by comparing the equipment and appraiser standard deviations to the total measurement error or total standard deviation. These ratios are used to see how significant the different effects of the equipment, appraiser and product error variations are on the total measurement system.
Table 1 shows a summary of all the calculations
used in this method, but only one standard deviation is used,
instead of finding a 95% confidence interval, as in the AIAG
Unfortunately, I find two errors in the AIAG R&R study. The first is a minor incorrect calculation of the part or product variation—in other words, there should be a correction factor that accounts for the variation induced by the measuring equipment. If this correction factor (although it may be very small) is not figured into the calculation, then equipment variation would be counted twice in the total variation.
The second and most significant error is the
final variation ratios—percent equipment variation (%EV),
percent appraiser variation (%AV) and percent part variation
(%PV). These are calculated using standard deviations instead of
variances (see the second column in Table 1). The results
obtained exaggerate the proportional effects of the equipment,
appraiser and part variation, as shown in the second column of
Therefore, this incorrect type of study cannot provide an index of whether the components of the measurement process are capable for the part or product under study.
The Correct Calculation
The measurement equipment error and part variation can be related by the following equation, assuming measurement error is independent of the part variation within a range defined by the natural process limits for the specified product:
This relationship can also be represented by a right triangle, as shown in Figure 1. For example, if σm = 5, σe = 3 and σp = 4, then 32 + 42 = 52,
This is a simple illustration of some of the misleading results for the final variation ratios in the AIAG method. Thus, a unit change in either the true product standard deviation (σp) or the standard deviation for measurement equipment error (σe) will not result in a unit change in the standard deviation for the actual product measurements (σm). On the other hand, one unit change in the true product variance or measurement equipment error variance will respond to one unit change in the variance of actual product measurement , because .
This shows part of the results of the gage R&R report in the AIAG method is incorrect. Therefore, a person doing the study should use variances to find the true individual percentage R&R ratio values instead of the standard deviations, where:
It should be noted that d2* values should be
used when k = 1 (see first row of Table 3), because of the
significant difference from assuming k>25.
The Results of Different R&R Studies
Table 2 shows the results of using different gage R&R studies for the same set of measurement data, where the analysis of variance (ANOVA) method is the most accurate, since it uses all the data, not just the ranges. The last column (IV) shows the correct percentage variations calculated from the new method I’ve partly described. It should be noted that with the new method, the repeatability, reproducibility and part variation percentages add up to 100%, which is not true with the AIAG method.
Comparing the AIAG R&R method with the ANOVA method (for the random effects model) and the new method, we can see the AIAG method exaggerates the effect of the %EV, %AV and %PV. That is, the %EV, %AV and %PV as obtained in the AIAG R&R study are incorrectly greater than the actual results, as shown in the second column of Table 2. This is due to the incorrect approach of comparing the standard deviations to the total standard deviation vs. the correct approach of using the additive law of variances.
This mistake in the AIAG R&R study could lead to a conclusion that the parts of the measurement process are incapable when they are actually capable.4 Also, mistakes such as these may mislead an organization to try to improve a capable measurement process, when it should actually reduce the process or product variance. The results of the ANOVA method and new method, however, agree closely. This can be seen in columns three and four of Table 2.
Change Is Needed
AIAG R&R methods may be misleading and should be modified. Reliable measurement data and analysis of those data are important. Part one of this column will help conscientious organizations further improve the quality of their products and the productivity of their processes.
In part two, I will introduce the appraiser variation (AV) as the third component of the total product measurement variation and will give an example of a complete and correct gage R&R measurement study.
- L.A. Brown, B.R. Daugherty and V.W. Lowe, Measurement Systems Analysis, third edition, Auto Industry Action Group, 2003.
- Acheson J. Duncan, Quality Control and Industrial Statistics, fifth edition, Richard D. Irwin Inc., 1986.
- Donald S. Ermer and P.E. Prond, “A Geometrical Analysis of Measurement System Variations,” ASQC Annual Quality Congress Transactions, 1993.
- Donald S. Ermer, “Pythagorean Theorem to the Rescue or Reliable Data Is an Important Commodity,” The Standard, ASQ Quality Measurement Division, Winter 2000/Spring 2001.
- Jay Bucher, The Metrology Handbook, ASQ Quality Press, 2004.
- Ronald D. Snee, “Are You Making Decisions in a Fog?” Quality Progress, December 2005.
- Philip Stein, “All You Ever Wanted to Know About Resolution,” Quality Progress, July 2001.
DONALD S. ERMER is the Procter & Gamble professor emeritus of total quality at the University of Wisconsin-Madison, where he also earned his doctorate in mechanical engineering. Ermer is an ASQ Fellow, is chair of Madison Section 1217 and received ASQ’s 1997 Edward Oakley Award and 2000 National Grant Medal.