## 2019

**Careful Interpolation Yields Useful Information**

#### Tips for getting extra resolution in measurements

* by***
Philip Stein**

What's between 2 and 3?

Well for starters, there's 2.1, 2.2, and so forth. Then there's 2.01, 2.02 and their fellows, and so on and so on. We metrologists and our fellow engineers, scientists and technicians are called upon to make many measurements a day, and a lot of them consist of dial, meter or pointer readings against a scale. I do my best to estimate readings between dial divisions--a practice often called interpolation--thus adding another digit of resolution beyond that engraved on the scale.

Early in my career I was taught to record my data with only the resolution that could be supported by the measurement process. So if an instrument had a precision of one part in a thousand, or 0.1%, the recorded result should only have three digits of resolution. In recent years, I have strongly recommended carrying one extra digit of resolution in readings because it makes the statistics of the results come out better. This is true even if the extra digit is completely random, but often, as with a "3-1/2 digit" voltmeter, that fourth number has some meaning.

In my travels as a consulting metrologist, I have found many people who are reluctant to interpolate at all. Some of them will round down or truncate, effectively ignoring any white space to the left of the needle. Others will round at the halfway point, recording the reading as the higher or lower scale division, depending on which side of the center the needle is found. Many of these advocates of low resolution actually expressed strong negative feelings when I asked them to interpolate, and many said that they had been taught never to use any judgment as to the meaning of a pointer reading between two divisions.

Well, it makes no sense at all to ignore an indication that is located between markings. In fact it's poor metrology. The indication is there, and it has real meaning. The obstacles to correct and useful inter- polation are that extra training, effort and experience may be required to get the most information out of these data.

When I worked at the National Bureau of Standards (NBS), before
it was renamed NIST, we were told of a study in which it had been
clearly demonstrated that a trained operator could meaningfully
interpolate 20 (!!) positions between two well-spaced divisions,
and that for a while people were being trained for that purpose.
After some research and help from colleagues, I now have a copy
of that study, "Scales and Reading Errors of Electrical Indicators,"
by Frank D. Weaver, of NBS, reprinted from *Instruments and
Automation*, November 1954.

Weaver says that a high-grade instrument should have an anti-parallax mirror and a knife-edge pointer with the knife-edge width about equal to that of the scale markings and preferably not more than one-tenth of the distance between two successive scale division marks. This is still good advice today, although with digital instruments so common it's no longer universally applicable.

After consulting and quoting some of the literature, Weaver conducted a study with experienced scale readers at NBS who tried to visually divide the scale readings of an electrical meter into 20 equal units between marked divisions. Out of 945 estimations, only 23 were in error by as much as 0.15 divisions, and only 123 readings were in error by as much as 0.1 divisions. The use of a hand-held magnifier materially aided the accuracy of estimation. Other sources of error contributed more than this to the overall measurement uncertainty.

The bottom line for us is that readers of pointer-type scales should interpolate to a tenth of a division, especially if they've had a bit of practice and some experience. They're very likely to add considerable statistical significance to the data that result. Of course scales with verniers have an advantage here and may easily be read to a tenth of a division. Here are some detailed tips for how to learn to interpolate:

**1. Understand parallax:** If your eye is not lined up with
the pointer, you may see the scale at an angle and misinterpret
what you see. Some scales have mirrors in order to help prevent
parallax. Move your head until the reflection of the needle is
hidden behind the needle itself, and then make a reading while
holding still. Prac-tice on a mirrored scale will help you reduce
the effect of parallax when you're taking readings without a mirror.

**2. Train and practice:** Use a protractor or a computer
drawing pack- age to make scales with practice readings at various
positions between divisions. Make sure your set of scales has
at least one reading for each of 20 equally spaced pointer positions.
Use them like flash cards, first revealing the answer after every
meas- urement, then testing yourself with groups of 10 readings
in a row. Training in a group where everyone reads the same drawing
ensures that everyone who uses the same set of instruments will
interpolate in the same way. Working in a group also greatly increases
confidence as everyone discovers that uniform, repeatable readings
are possible.

**3. Use statistics:** Usually, over a group of several dozen
or more measurements, there should be approximately the same number
of each digit, 0 through 9, in the last place. It's well known
that untrained people tend to prefer certain digits during interpolation,
but trained people should have overcome this problem.

Plot a histogram or dotplot of the last digits, or plot all the data using a stem-and-leaf plot. If the distribution looks very nonuniform, or overly uniform, you may use the Kolmogorov-Smirnov test to confirm your suspicions. Be careful, though, because some measurement activities, such as repeated measurements of a standard, may always give answers that vary only one or two digits in the interpolated place.

The answers in this case are not expected to be uniformly distributed. Check your statistics while measuring unknowns that could be expected to give varying answers.

Now what about digital meters? There's no pointer, and no ambiguity about the reading, but there might not be as many digits of resolution as are needed to extract the full meaning from the data.

If your digital indicator is limited in how many digits it can display, you can always take the average of multiple readings. This is the digital equivalent of visually interpolating between scale markings. If, for example, you have 10 readings--1.3, 1.2, 1.3, 1.3, 1.3, 1.2, 1.2, 1.3, 1.2 and 1.3--you have six of 1.3 and four of 1.2 with a total of 12.6. By taking the mean of 1.26, you can add another digit of resolution to your process.

The standard error of this mean, or standard deviation, is only a little more than three times narrower than the standard deviation of the individual readings, though. Taking an average of multiple independent measurements yields a standard deviation equal to the standard deviation of each measurement divided by the square root of the number of measurements in the average, so you add less precision than resolution by this procedure. Nevertheless, if averaged multiple measurements are cost effective, they are an extremely good idea.

So why is it useful, even important, to have and carry that extra digit? This is a mixed bag of reasons, so please bear with me.

**1. Is there some information in the extra digit?** We spoke
above of a 3-1/2 digit digital meter. This almost always means
that the largest number that can be displayed is 1999, not just
999, for a resolution of one part in 2000. If the precision of
a measurement made by that meter is better than one part in 1000,
the extra half-digit is not purely random but carries some real
information. Better keep it around.

**2. Will the result be used in a calculation?** Care needs
to be taken to make sure that the arithmetic doesn't round off
intermediate results. It's possible to lose significant digits
from your answer.

**3. Will the results be displayed or analyzed statistically?**
Even such simple tools as histograms and control charts work best
if the distribution of the data they display is normal or Gaussian
(bell-shaped curve). In cases when high-speed electronically collected
data have too little resolution, some practitioners actually add
a bit of random noise (dithering) to make the distribution more
palatable to the analysis.

Actually, there's no harm in carrying around a bit of extra resolution as long as the limitations of the results are understood and respected when the final answer is used. In the past, we made sure that there weren't extra digits as a means of communicating with the end user of the data. Today, we realize that keeping a bit of extra precision around is almost always a good idea.

PHILIP STEIN *is a metrology and quality consultant in private practice
in Pennington, NJ. He holds a master's degree in measurement science
from The George Washington University, in Washington, and is an ASQ
Fellow. For more information, go to www.measurement.com.
*

If you would like to comment on this article, please post your remarks on the Quality Progress Discussion Board, or e-mail them to editor@asq.org.

Featured advertisers