MEASURE FOR MEASURE
The dangers of oversimplifying measurement data
by Dilip Shah
If you put your head in the oven and your feet in a bucket of ice, you will feel fine on average. That’s an old statistician’s joke, but it’s no laughing matter when crunching and reporting measurement data, whether it’s for measurement uncertainty or a calibration report.
Averaging measurements is useful. It’s better to take more than one measurement and report the average of the results. When you take a single measurement, it may be fine, or it may be wrong if the measurement is taken or written down erroneously. We call this single-measurement bliss.
But bliss is often ignorance in disguise and dangerous if the single measurement is incorrect. Therefore, it’s better to take more than one measurement and report the average.
But reporting the measurement average has certain flaws. It hides the extreme range of the measurements—such as our example of a head in the oven with feet in ice. So it’s a good practice to report the range (maximum value - minimum value) of the reported average or the standard deviation (measure of data dispersion) of the measurement results.
In the era before computers, it was much easier and faster to calculate the range to identify extreme values. But in the era after digital, calculating standard deviation is very easy thanks to software or spreadsheets.
For measurement uncertainty, Type A uncertainty data, such as repeatability, are reported as a sample standard deviation of a series of measurements. The equation for the sample standard deviation is:
The Guide to Uncertainly for Measurement (GUM) has another equation I’ve seen used for calculating standard uncertainty in many measurement uncertainty budgets I’ve reviewed.
When I question the measurement uncertainty claims of those budgets, most people will point to the GUM and say that because it’s in the GUM, it’s right. Yes, it’s in the GUM, but their application of this equation may be incorrect.
The other standard deviation equation in the GUM that is referred to is for the standard deviation of the mean (SDOM), which is sometimes referred to as the standard error of the mean. The equation for the SDOM is:
The SDOM—using the sample average as a method of estimating the population average—is the standard deviation of those sample averages over all possible samples of a given size drawn from the population. The SDOM also refers to an estimate of that standard deviation computed from the sample of data being analyzed.
Measurement data are often summarized using the mean (average) and standard deviation of the mean with the SDOM. This can lead to confusion about their interchangeability. But the mean and standard deviation are descriptive statistics, whereas the mean and SDOM describe bounds on a random sampling process.
Despite the small difference in equations for the standard deviation and the SDOM, this small difference changes the meaning of what is being reported from a description of the variation in measurements to a probabilistic statement about how the number of samples will provide a better bound on estimates of the population mean in light of the central limit theorem.
For those in the metrology profession, this is best illustrated with a measurement uncertainty example.
For repeatability data (Type A uncertainty), 10 individual measurements are made as shown in Table 1 with the sample standard deviation calculated. The uncertainty budget using these individual measurements for repeatability data (Type A uncertainty) and other contributors (in this example, resolution and accuracy of the standard) is shown in Table 2.
If the same 10 measurements for Type A uncertainty data were calculated using 10 averages of 10 individual values, the SDOM is calculated as shown in Table 3. The uncertainty budget using these 10 averages and other contributors (in this example, resolution and accuracy of the standard) is shown in Table 4.
Pick your approach
The difference in percentage contribution due to repeatability is significant depending on which method of calculating standard deviation is used. In many cases, the Type A data (repeatability) collected for measurement uncertainty are individual values. This is due to the time constraints.
If the SDOM is used for these individual measurements, the uncertainty contribution due to repeatability may be underreported, while the laboratory’s actual measurement uncertainty for that particular range and parameter may be higher.
In some cases, the repeatability may be much smaller than the actual resolution of the device. The actual performance of the device (or a measurement standard) may not meet the measurement uncertainty claimed because of the incorrect use of the appropriate standard deviation.
To summarize: For individual repeated measurements, use sample standard deviation. For individual averages of measurements, use standard deviation (error) of the mean.
By taking repeated measurements and averaging versus using a single measurement—as well as calculating the appropriate standard deviation—ignorant bliss is replaced by more confidence in your measurements.
International Organization for Standardization, ISO/IEC Guide 98-3:2008—Guide to the expression of uncertainty in measurement.
Shah, Dilip, "Keep your Resolution," Quality Progress, March 2011, pp. 56-58.
Shah, Dilip, "Student Teaching," Quality Progress, January 2012, pp. 47-49.
Dilip Shah is president of E = mc3 Solutions in Medina, OH. He is chair of ASQ’s Measurement Quality Division and past chair of Akron-Canton Section 0810, and is co-author of The Metrology Handbook (ASQ Quality Press, 2004). Shah is an ASQ-certified quality engineer and calibration technician, and a senior member of ASQ.