﻿ How Statistics Can Improve Mensuration - ASQ

# How Statistics Can Improve Mensuration

By Lloyd S. Nelson

COUNTERINTUITIVE facts are always interesting. In thinking of a title for this piece, I considered "How Statistics Can Enable a Measuring Device to Yield More Precision Than It Was Designed to Provide". This is accurate, but far too long. The title used instead is short enough, but gives no clue as to what is involved. To nd out, read on.

When my younger daughter was in the first grade, she had a ruler that bore no graduations other than inch marks. I appropriated it to use as a prop for an interesting experiment in my statistics classes. It went as follows.

Given that the one-inch marks on this ruler are exactly positioned and are infinitely thin, how precisely could a rod, say exactly 5 inches long, be measured? Of course, following the usual practice, one would read the integral number of inches and add the estimated fractional part of the length. Many class members thought that the result could be successfully accomplished to within 1/4 inch. Some were of the opinion that with care it could be done to within 1/8 inch. A few extremists claimed that great care could produce a result correct to within 1/10 inch. No one thought that the error could be as small as 1/100 inch.

There was surprise accompanied by disbelief when I told them that the error could be as small as 0.01 inch, and indeed could be even smaller! In fact, it could be (in principle) as small as one wished. How could such a counterintuitive result be obtained? Statistics comes to the rescue.

Lay the 5-inch rod on the ruler and note how many inch marks are covered. Depending on where it is laid, the number will be either 5 or 6. Here a demonstration is most helpful. Now, imagine repeatedly laying it down randomly and recording the number of inch marks covered. The arithmetic mean of these inch mark numbers approaches a limit that is the true length―which here is 5 inches. This is analogous to the repeated tossing of a coin to estimate the probability of obtaining a head.

To express this algebraically, let a equal the number of times the smaller number of inch marks is covered, and let b equal the number of times the larger number of inch marks is covered. Then, an estimate of the true length of the rod is with, as shown by Russell (1956) , a maximum standard error of , where n is the total number of trials = . Toachieve a standard error equal to or less than (say) 0.01, the total number of trials required is 2500. This might be feasible if it could be done mechanically or electronically.

The example in which the length is halfway between two inch marks seems to be the best to motivate the general case. In concluding the exercise, I pointed out that the ner the graduations on a ruler the greater the possible precision. But the existence of these ne divisions can also be viewed as simply away of saving time and e ort. This procedure is treated in more detail and extended to the measurement of at surfaces, spheres, and solid angles by Russell (1956).

Key Words: Approximation, Count Data, Estimation, Mean, Precision, Replicate Measurement Data, Sampling .. 