It’s Not Always What You Say, But How You Say It
by Lynne B. Hare
When my daughter Jennifer was in third grade, her teacher, Mrs. M., gave her an assignment to divide the number 12: first by six, then by four, then three, two, one and zero. She came to me and said, “Daddy, I can do them all but the last one. How many times will zero go into 12?”
“Hold out your hand,” I said. “Now I’m going to put nothing in it. Now I’m going to do it again. How long do you suppose I can keep doing that?”
“Forever!” she exclaimed.
“Then that’s how many times zero will go into 12.”
Jennifer put “forever” in the blank provided on Mrs. M.’s homework sheet. Two days later, she got her paper back with that one marked wrong. She bristled. Ever since I’ve known her, she has spoken truth to power. True to form, she accosted Mrs. M. after school. Mrs. M. said the answer was zero. Jennifer started to explain how it wasn’t: “Here, Mrs. M.,” she said, “hold out your hand.” Mrs. M. was having none of that.
Jennifer, unwilling to accept a rebuke, played her trump card. As Mrs. M. was heading back toward the front of the room, Jennifer said, “The answer is ‘forever.’ My daddy is a numbers doctor, and he said so!”
Saying It the Right Way
Being right isn’t always enough. Quite often, the way we are right makes the difference. Evidence to that effect is taken from Jack Youden, beloved chemist and statistician who served at the National Bureau of Standards, now the National Institute for Standards and Technology, from 1948 until his death in 1971. As his colleagues attest, he seemed always to know the right thing to say to persuade others toward his point of view.
Among his many contributions to the statistical body of knowledge is the Youden plot, which has proven extremely useful in the analysis and interpretation of data generated by interlaboratory studies. For these studies, Youden recommended that two nearly identical samples be prepared, divided and sent to each of the participating laboratories. The resulting data was used to check laboratory alignment. Table 1 shows a sample data set. The tabled numbers are scaled from an actual interlaboratory study of vitamin A analysis.
While the study design is simple enough, Youden was disappointed with statistical statements based on the analysis of variance, calculation of variance components and other summary figures typically generated by statisticians. Not that statisticians’ hearts aren’t in the right place—perish the thought!—but chemists just didn’t understand what the statisticians were talking about.
His invention, which we now call the Youden plot, is simply constructed. On a sheet of graph paper—remember that?—mark off convenient increments spanning the range of the data, first on the horizontal axis and then the same on the vertical axis.
Each laboratory’s plotted point is the intersection of its first replicate value on the horizontal axis and its corresponding second replicate value on the vertical axis.
Therefore, the graph will show as many plotted points as there are laboratories. Figure 1 displays the data in Table 1 and the features of the Youden plot.
In Figure 1, the proximity of the laboratory’s plotted point to a diagonal line with slope 1 and intercept zero shows agreement of replicates. Notice that laboratory M, and, to a lesser extent, laboratory E, show high variation between duplicates. From his writings, it is clear Youden would share these results confidentially to the respective laboratories and offer them a second opportunity. That’s good statistical bedside manner.
Youden’s Theory Resonates With Modern Statisticians
The scientific world is better for having theoretical underpinnings for all tools and techniques.
In his unpublished manuscript, “Youden’s Legacy in Metrology,” my friend, Mark Vangel, assistant professor in radiology at Harvard Medical School and statistician at Massachusetts General Hospital, pointed out, “Youden would not have much use for this [following] theory, but it is interesting, and it can aid in the construction and interpretation of the plots.
“We model our measurements as follows with two different means for the two replicate materials and a common random laboratory effect. Within and between-lab components of variance are assumed to have normal distributions. The p pairs of measurements can then be regarded as a sample from a bivariate normal distribution.
“From the eigenvectors of the covariance matrix, we see that the points should fall in an ellipse tilted at a 45° angle. With a little further effort, we can construct confidence and prediction ellipses, and we can show that the squared ratio of the major to the minor axes of these ellipses is precisely the F-statistic for the hypothesis that the between-lab variance is zero.”
It is gratifying that Youden’s insight and the modern statisticians’ theory agree.
—Lynne B. Hare
Youden also needed a device to show deviations from the central value by more than chance alone would allow. His choice of devices was the circle centered at the intersection of the replicate medians after obvious outliers were removed. (I used the means calculated with results of laboratory M removed.) The radius of the circle is based on a multiple of the within-laboratory standard deviation, depending on the desired percentage of observations anticipated to fall within a bivariate normal distribution.
Youden recommended that a multiple of 2.5 to 3 be used to contain most of the plotted points if there were no differences among laboratories, and he wasn’t too picky about the exact multiple. (I used a multiple of 2.45 times the pooled within-laboratory standard deviation of 7.48 to arrive at a radius of 18.3 following one of his examples: 2.45 standard deviations should contain about 95% of the plotted points assuming no differences among laboratories.)
Laboratories D, H and N show reasonable agreement between replicates, but also clear departure from the expected value based on the combined findings from all the laboratories, M excluded. Again, Youden demonstrated excellent statistical bedside manner by making individual copies of the plot with the identity of the errant laboratory hidden to all but it.
“When the points lie closely along the 45° line, the conclusion may be drawn that many of the laboratories are following rather carefully their own versions of the test procedure,” Youden wrote. But when points lie outside the circle, “The laboratories responsible for these points almost certainly have somehow got substantial systematic errors incorporated in their techniques.”1
This is better than saying the analysis of variance followed by the multiple comparisons test shows that laboratories D, H and N are in need of further work. The point is that Youden found a way the client would understand and would be motivated to action.
The world will always have its share of Mrs. M.’s, but for people willing to listen, a simple, graphical approach that avoids jargon will win the day.
- William John Youden, “Graphical Diagnosis of Interlaboratory Test Results,” Industrial Quality Control, May 1959, Vol. 15, No. 11. Reprinted in Precision Measurement and Calibration—Statistical Concepts and Procedures, Special Publication 300, Vol. 1, Feb. 1960.
- Cornell, J.A., “Youden Address—Remem-bering Jack Youden,” ASQ Statistics Division Newsletter, Winter 2007.
- Vangel, Mark G., “Youden’s Legacy in Metrology,” unpublished manuscript, ASQ Statistics Division Fall Technical Conference, October 2000.
LYNNE B. HARE is director of applied statistics at Kraft Foods Research and Development in East Hanover, NJ. He received a doctorate in statistics from Rutgers University, New Brunswick, NJ. Hare is a past chairman of ASQ’s Statistics Division and a fellow of both ASQ and the American Statistical Assn.