What to include

Q: We have lots of different instructions, visual aids, training documents and standard operating procedures that aren’t part of our quality management system (QMS). Where do you draw the line on what really needs to be in the QMS and controlled?

John Leathers
Ferndale, WA

A: There are two considerations. First, ask yourself whether the the visual aids, training documents and standard operating procedures fall within the scope of the ISO 9001 or ISO 13485 standards. If so, they should be controlled.

Second, the interpretation of the previous paragraph will be a function of the industry. If the product involved is a medical device or pharmaceutical, interpretation of "what we need to control" is typically much stronger toward putting "everything" under document control.

To put it another way: If the document was important enough to create, somebody needs it, and it should be controlled so it is kept current and approved by the right people. If the product is simpler and less regulated, there is reduced interpretation on the depth of what should be controlled.

Peter E. Pylipow
Senior design excellence engineer
Vistakon—Johnson and Johnson Vision Care
Jacksonville, FL

For more information

Prediction vs. tolerance

Q: When would you build a prediction interval instead of a tolerance interval?

Erika Marceles
Tampa, FL

A: Prediction intervals and tolerance intervals are certainly related, but seek to answer different questions. A prediction interval answers the question about a new observation and what its observed value is likely to be, while a tolerance interval provides a range of values that characterize percentiles of the population distribution of interest.

The tolerance interval requires an additional quantity to be specified: what proportion of the population distribution is to be captured in the interval. The prediction interval considers what likely values are of future observations, while the tolerance interval summarizes what we know about the current population distribution. Table 1 highlights some of the key differences and similarities between the two types of intervals.

Table 1

Consider an example in which production data are readily available for the weights of a particular type of parts. A 95% prediction interval would provide an answer for the range of values in which you are quite confident that a new observation would fall. This would be helpful if you were going to provide a future part to a customer who wanted to know what weight was likely.

On the other hand, if you’re interested in characterizing the distribution of weights for that particular population of parts, then a tolerance interval would be more appropriate. A 95% tolerance interval for 50% of the parts would give a range of values for which you are quite confident that at least half of all the parts coming out of that production process would fall within the specified range.

Selecting the right interval to use to answer the question of interest is just as important as using the right equation to calculate the interval limits themselves.

Christine Anderson-Cook
Research scientist
Los Alamos National Laboratory
Los Alamos, NM

For more information

  • Anderson-Cook, Christine, "Interval Training," Quality Progress, October 2009, pp. 58–60.
  • Hahn, Gerald J., and William Q. Meeker, Statistical Intervals: A Guide for Practitioners, Wiley-Interscience, 1991.


Q: I have performed a full factorial experiment with a random effects model that includes the following characteristics: three factors, four samples (levels) per factor and two replicates. I analyzed the results using a three-way balanced analysis of variance (ANOVA) and obtained the variance components of each of the significant factors.

I have been asked to provide rationale regarding the selection of the sample size (levels) and replicates to provide a power of the test or a confidence interval for the variance components. My selection of sample size (levels) and replicates was based mainly on the amounts of runs that had to be performed.

I have gone through a lot of literature, but I have not found a power of the test analysis for a three-way ANOVA with four levels. Any guidance you could provide would be greatly appreciated.

Mauricio Rodriguez
L’Île Bizard, Quebec

A: There are no easy answers to this query, but there are probably ways to validate the design and analysis.

Power and Precision Software (www.power-analysis.com) has an effective program that produces power analyses for three-way ANOVA designs and can be used free for 30 days. While it may not have the precise specifications of the design outlined in the question, it will be sufficient and can give either the exact power of the results or, alternatively, the required sample size.

You could also try using one of the many free ANOVA power online calculators:

Also, when asked about the power of a test, it is important to remind the reviewer that if the test was significant, there was sufficient power to show a difference. Only when a result is not significant is the question about sample size and power important. That’s because we are interested in knowing if there was sufficient power and sample size to show a difference if a difference existed.

I. Elaine Allen
Professor of statistics and entrepreneurship
Babson College
Wellesley, MA

For more information

Average Rating


Out of 0 Ratings
Rate this article

Add Comments

View comments
Comments FAQ

Featured advertisers