Three Robust Scale Estimators to Judge Unreplicated Experiments - ASQ

Three Robust Scale Estimators to Judge Unreplicated Experiments

Results from unreplicated experiments can be evaluated by testing of standardized single degree of freedom contrasts with a robust estimator of their standard error constructed from the same contrasts. This paper presents constants to calculate three estimators, and critical values to test the contrasts. The constants and the critical values depend on the number of contrasts. The paper covers all contrast numbers in the range from 7 to 127. It is proposed to make the choice between the estimators according to the resolution of the design.

Keywords: Design of Experiments, Half-Normal Plots, Standard Error

by Eric D. Schoen and Enrico A. A. Kaul, TNO Institute of Applied Physics, 2628 CK Delft, The Netherlands


Results from unreplicated experiments are commonly evaluated by making half-normal or normal plots of absolute valued single degree of freedom contrasts. Half-normal plotting was introduced by Daniel (1959). He plotted absolute valued contrasts from two-level experiments against expected quantiles of the half-normal distribution. If the experimental error is normal and if there are only a few active contrasts, then most will come from a normal distribution with zero mean and the same unknown variance. Inactive contrasts will then roughly lay on a straight line through the origin in the plot. Those not compatible with this line are designated active. Thus half-normal plots will separate the few active contrasts from the inactive ones.

Daniel (1976) pointed out that plotting signed contrasts against expected quantiles of the normal distribution will also separate active from inactive contrasts. Because such plots are more capable of detecting anomalies in the data, Daniel (1976) recommends full normal plots.

The subjectivity as to which contrasts deviate from the straight line is a well-known problem with (half-)normal plotting. To overcome this problem, various robust estimators of the contrasts’ standard errors have been developed; see Haaland and O’Connell (1995) for an overview. Haaland and O’Connell define a family of estimators constructed from the ordered absolute values of the contrasts, by the following common elements. First, an initial estimate of the standard error is calculated as a quantile of the full set of ordered absolute valued contrasts, multiplied by a consistency constant determined from the normal distribution. Second, potential active contrasts are stripped from the others by retaining only those smaller than a constant times the initial estimate. Third, a scalar function of the remaining contrasts is multiplied with a simulated consistency constant to give the final estimate of the standard error. Based on simulation results for 15 contrasts, Haaland and O’Connell recommend three estimators according to a priori ideas on the likely number of active contrasts (1–3, 4–6, and 7–8, respectively). One of the three recommended estimators is based on the median of the full set and the root mean square of the retained set of contrasts. This so-called adaptive standard error (ASE) was originally proposed by Dong (1993). The other two estimators are based on the median or the 0.45 quantile of the full set, respectively, and the median of the retained contrasts; these are pseudo standard errors (Lenth (1989)), and the two versions will be designated PSE(50) and PSE(45), respectively. In general, the ASE is less robust against contamination with active contrasts than the PSE(50), because it uses all the contrasts below the cut-off point. The PSE(50) is obviously less robust than the PSE(45).

Haaland and O’Connell suggest judging ratios of the contrasts and the estimated standard error against critical values determined by simulation. Their paper has consistency constants to calculate ASE and PSE(50), and critical values to test with the PSE(50), each for experiments with 7, 11, 15, 17, 23, and 31 contrasts. The present paper extends the results to all contrast numbers in the range from 7 to 127, and all three estimators. It also offers a guideline for choosing among the estimators. The tables in this paper show the results for contrasts from 7 to 96, 112, 120, 124, and 127. The remaining results can be obtained from the authors.

The extension of the Haaland and O’Connell results to all contrast numbers in the chosen range is motivated by the great variety of possible designs. Two-level designs with randomization restrictions, for example, have their 2n – 1 contrasts divided over sets of different precision. Specifically, split-plot and strip-block arrangements (Box and Jones (1992)) result in a division over two and three sets of contrasts, respectively, while an extension of the latter arrangement (Miller (1997)) has a division into four sets of contrasts. The contrast numbers of at least one of the sets will not equal 2n – 1. Further, the Plackett-Burman designs have contrast numbers equalling 4p – 1, p being an integer ranging from 2 up to 25. Haaland and O’Connell do not cover contrast numbers 19 and 27, or any number above 31.

Finally, when factors other than two-level ones are included, the total number of contrasts may equal neither 2n – 1 nor 4p – 1. More specifically, Box and Draper (1987) and Draper et al. (1994) illustrate the break down of results from central composite and Box-Behnken experiments, respectively, into single degree of freedom contrasts.

We extend the Haaland and O’Connell results to a wider range because it is not uncommon to have experiments with larger numbers of contrasts than 31. The Box-Behnken design of Draper et al. (1994), for example, has 46 runs. Schoen (1997) used a 128-run two-level split-split-plot experiment to investigate the effect of 11 factors on properties of Dutch cheeses.

The remaining part of this paper is organized as follows. First, we define the estimators used and present the method of calculating the required constants. Subsequently, we present consistency constants and critical values for testing of the contrasts with Type I errors of 0.20, 0.15, 0.10, 0.05, and 0.01, respectively. Further, we propose a guideline for choosing among the three estimators. A practical example concludes the paper.

Read Full Article (PDF, 118 KB)

Download All Articles

Featured advertisers

ASQ is a global community of people passionate about quality, who use the tools, their ideas and expertise to make our world work better. ASQ: The Global Voice of Quality.