Results from unreplicated experiments can be evaluated by testing of standardized single degree of freedom contrasts with a robust estimator of their standard error constructed from the same contrasts. This paper presents constants to calculate three estimators, and critical values to test the contrasts. The constants and the critical values depend on the number of contrasts. The paper covers all contrast numbers in the range from 7 to 127. It is proposed to make the choice between the estimators according to the resolution of the design.

Keywords: Design of Experiments, Half-Normal Plots, Standard Error

*by* **Eric D. Schoen** and **Enrico A. A. Kaul,
TNO Institute of Applied Physics, 2628 CK Delft, The Netherlands**

*INTRODUCTION*

Results from unreplicated experiments are commonly evaluated by making half-normal or normal plots of absolute valued single degree of freedom contrasts. Half-normal plotting was introduced by Daniel (1959). He plotted absolute valued contrasts from two-level experiments against expected quantiles of the half-normal distribution. If the experimental error is normal and if there are only a few active contrasts, then most will come from a normal distribution with zero mean and the same unknown variance. Inactive contrasts will then roughly lay on a straight line through the origin in the plot. Those not compatible with this line are designated active. Thus half-normal plots will separate the few active contrasts from the inactive ones.

Daniel (1976) pointed out that plotting signed contrasts against expected quantiles of the normal distribution will also separate active from inactive contrasts. Because such plots are more capable of detecting anomalies in the data, Daniel (1976) recommends full normal plots.

The subjectivity as to which contrasts deviate from the
straight line is a well-known problem with (half-)normal plotting.
To overcome this problem, various robust estimators of the
contrasts’ standard errors have been developed; see Haaland
and O’Connell (1995) for an overview. Haaland and O’Connell
define a family of estimators constructed from the ordered
absolute values of the contrasts, by the following common
elements. First, an initial estimate of the standard error
is calculated as a quantile of the full set of ordered absolute
valued contrasts, multiplied by a consistency constant determined
from the normal distribution. Second, potential active contrasts
are stripped from the others by retaining only those smaller
than a constant times the initial estimate. Third, a scalar
function of the remaining contrasts is multiplied with a simulated
consistency constant to give the final estimate of the standard
error. Based on simulation results for 15 contrasts, Haaland
and O’Connell recommend three estimators according to
*a priori* ideas on the likely number of active contrasts
(1–3, 4–6, and 7–8, respectively). One of the
three recommended estimators is based on the median of the
full set and the root mean square of the retained set of contrasts.
This so-called adaptive standard error (ASE) was originally
proposed by Dong (1993). The other two estimators are based
on the median or the 0.45 quantile of the full set, respectively,
and the median of the retained contrasts; these are pseudo
standard errors (Lenth (1989)), and the two versions will
be designated PSE(50) and PSE(45), respectively. In general,
the ASE is less robust against contamination with active contrasts
than the PSE(50), because it uses all the contrasts below
the cut-off point. The PSE(50) is obviously less robust than
the PSE(45).

Haaland and O’Connell suggest judging ratios of the contrasts and the estimated standard error against critical values determined by simulation. Their paper has consistency constants to calculate ASE and PSE(50), and critical values to test with the PSE(50), each for experiments with 7, 11, 15, 17, 23, and 31 contrasts. The present paper extends the results to all contrast numbers in the range from 7 to 127, and all three estimators. It also offers a guideline for choosing among the estimators. The tables in this paper show the results for contrasts from 7 to 96, 112, 120, 124, and 127. The remaining results can be obtained from the authors.

The extension of the Haaland and O’Connell results
to all contrast numbers in the chosen range is motivated by
the great variety of possible designs. Two-level designs with
randomization restrictions, for example, have their 2^{n}
– 1 contrasts divided over sets of different precision.
Specifically, split-plot and strip-block arrangements (Box
and Jones (1992)) result in a division over two and three
sets of contrasts, respectively, while an extension of the
latter arrangement (Miller (1997)) has a division into four
sets of contrasts. The contrast numbers of at least one of
the sets will not equal 2^{n} – 1. Further,
the Plackett-Burman designs have contrast numbers equalling
4*p* – 1, *p* being an integer ranging from
2 up to 25. Haaland and O’Connell do not cover contrast
numbers 19 and 27, or any number above 31.

Finally, when factors other than two-level ones are included,
the total number of contrasts may equal neither 2^{n}
– 1 nor 4*p* – 1. More specifically, Box and
Draper (1987) and Draper et al. (1994) illustrate the break
down of results from central composite and Box-Behnken experiments,
respectively, into single degree of freedom contrasts.

We extend the Haaland and O’Connell results to a wider range because it is not uncommon to have experiments with larger numbers of contrasts than 31. The Box-Behnken design of Draper et al. (1994), for example, has 46 runs. Schoen (1997) used a 128-run two-level split-split-plot experiment to investigate the effect of 11 factors on properties of Dutch cheeses.

The remaining part of this paper is organized as follows. First, we define the estimators used and present the method of calculating the required constants. Subsequently, we present consistency constants and critical values for testing of the contrasts with Type I errors of 0.20, 0.15, 0.10, 0.05, and 0.01, respectively. Further, we propose a guideline for choosing among the three estimators. A practical example concludes the paper.

Read Full Article (PDF, 118 KB)