A Powerful Analytical Tool - ASQ

A Powerful Analytical Tool


Download the Article (PDF, 90 KB)

Christopher Nachtsheim, University of Minnesota, and Bradley Jones, SAS Institute Inc.


The success stories compiled by early adopters of the Six Sigma approach to business problem solving are impressive, spanning industries from high tech manufacturing to service providers and companies from the Fortune 100 to fast food franchises.

Six Sigma’s power comes from its integration of the team based approach, customer orientation, financial motivation and assessment, tangible rewards for success, qualitative and statistical tools, and focus on short duration, high impact projects.

Six Sigma includes elements of W. Edwards Deming’s management and quality philosophy,(1) but it is more. At Six Sigma’s core is the fundamental cycle of engineering problem solving: plan-do-check-act.

Readers of Six Sigma Forum Magazine know the managerial and organizational aspects of change management are at least as important as the technical tools—thus the emphasis on top management involvement, the use of cross functional teams, problem selection and definition, then standardizing and institutionalizing the resulting improvements.

Correlation vs. Causation
Using statistical tools such as regression, a Six Sigma team can establish a correlation between variables of interest. But, correlation does not imply causation. A small example illustrates this important, often overlooked, point.

Programmers at a large insurance firm were recently given the option of attending a training workshop on object oriented programming, and about half the eligible programmers volunteered. Each year the productivity of all programmers is rated on a 10-point scale.

Figure 1 compares the ratings of the programmers who attended the training with those who did not. The supporting statistical analysis shows the observed difference in average productivity is highly statistically significant.

It is tempting to conclude the increase in productivity was caused by the training. But causality does not necessarily follow what was simply an observational study of historical data. In this case, review of the prior year’s productivity scores revealed the volunteers were already the most productive. So much for the effectiveness of the training.

So how do we establish causality? We must actively manipulate the variables we can control—as opposed to passively observing them—and quantify the effects. Because active manipulation of variables is built into design of experiments (DOE), it is the most powerful of the analytical tools in the kit of the Black Belt.

Let’s consider ways to apply DOE in each of the four phases in every Six Sigma project: identification, characterization, optimization and institutionalization (analogous to the traditional Six Sigma methodology of define, measure, analyze, improve and control).

The first step in any Six Sigma project is the identification step. Here the focus is on developing an understanding of how variation in internal processes affects business results and customer satisfaction. In a manufacturing situation, a progress metric might be the amount of scrap and rework produced. In a service operation, it might be the number of customer complaints per week.

The voice of the customer (internal or external) is critical in choosing progress metrics. If the measure isn’t on the customer’s radar screen, it probably shouldn’t be on yours. Designed customer surveys are a way to use DOE in the define step.

To establish price points for the standard package and four options, an automotive company designed a survey in the form of a five-factor experiment. In this way, the company was able to fix pricing to maximize revenue.

The company also found out which features customers perceived to have the most value. This guided the Six Sigma teams in choosing high impact process studies. Using DOE, surveys can also be fielded in ways to test for the effects of various customer demographic factors, such as age, gender and economic level.

Characterization—Measure and Analyze
The characterization step establishes the current performance levels for the key progress metrics and sets goals for those metrics. DOE is frequently used in this step to help determine process capability, compare alternative measurement methods and establish the validity of selected metrics.

To establish process capability and fully understand the current process, potential sources of variability, such as operators, shifts, raw material lots and ambient conditions, are systematically varied in a designed experiment. DOE can also be used to characterize the accuracy and precision of alternative measurement systems and determine, in a validation stage, whether what is being measured is truly linked to the customer’s perception of quality.

Optimization—Improve and Control
The optimization step is where the rubber meets the road in a Six Sigma project. Here, the team identifies the specific changes that will yield the desired improvements. In the words of Mikel Harry and Richard Schroeder, “Optimization identifies what steps need to be taken to improve a process and reduce the major sources of variation. The key process variables are identified through statistically designed experiments; Black Belts then use these data to establish what ‘knobs’ must be adjusted to improve the process.”(2)

DOE is the one tool that can effectively establish and quantify the causal relationships between variables that can be controlled (process steps, materials, equipment, training levels) and key process outputs (defect rates, on-time deliveries or scrap and rework.) When implemented, the improvements identified via DOE lead to a new, improved level of process capability. Using this new capability, the Six Sigma team can quantify the savings to the corporation.

In the following more complete DOE example for this step, the project resulted in savings of $5 million a year.

Institutionalization—Standardize and Integrate
A successful optimization step, as in our example, will identify a new set of best practices for the management of a critical to business process. It is important these best practices be standardized, communicated and implemented throughout the organization. This is the essence of the institutionalization step.

DOE is used within this step in a variety of ways. For example, the optimization step may have involved experimentation with a manufacturing process in a pilot plant or with a product receiving process in a particular distribution center. Scale up from a pilot plant setting to full production or from one distribution center to all such centers may require fine-tuning. Alternatively, it may be important to establish the sensitivity of key metrics to the control variables on a site-by-site basis. Designed experiments are the right tool for performing such sensitivity and fine-tuning studies.

Optimization Example
This simplified example was adapted from some recently reported Six Sigma studies that targeted the reduction of registration error.(3, 4)

Registration error is a frustrating and ongoing source of scrap in the production of dense, multilayered, printed circuit boards. Registration refers to the alignment of electrical connections that must occur between layers of circuits.

Warping, shrinkage, rotation or other movements that occur during the lamination process can lead to registration error. If the registration error falls outside the specification limits of -12 to +12, the board will be defective.

Factors that affect registration error can be divided into two groups: the constituents and operating conditions. The constituents are the epoxy and core materials, while the operating conditions include such process settings as oven pressures and temperatures.

Two objectives were specified in the identification step of this Six Sigma project: to reduce the scrap rates due to registration error and reduce cost through the use of lower priced materials. During the characterization step, it was noted that two suppliers had been competing. Previous historical data had indicated use of the low cost supplier’s epoxy led to high registration error rates. Using the current process, any potential savings from the use of the low cost epoxy were lost due to the associated high scrap rates. An experiment was designed with the goal of determining optimal process settings for each supplier’s epoxy.

Figure 2 shows the data table from the experimental runs. Note both temperature and pressure were tested at three levels. This was necessary to fit a quadratic response surface model to registration error. A full factorial design would take 18 runs. This was reduced to 14 runs through the use of a D-optimal design.

The model used to analyze the data in Figure 2 is entered into JMP’s fit model dialogue box, shown in Figure 3.(5)

Figure 4 shows predicted registration error at the current process settings for the high cost supplier. The process mean is 3.7 and the standard deviation is about 2.5. Recall that if the registration error falls outside the specification limits of -12 to +12, the board will be defective. The predicted scrap rate for this process (assuming a 1.5 standard deviation drift in the process mean—a standard Six Sigma calculation) is 3.44%.

Figure 5 shows that the mean registration error at the current process settings for the low cost supplier is about 14.4. The scrap rate for this process is well over 50%, which is clearly not a capable process.

By contrast, Figures 6 and 7 show the optimal settings of temperature and pressure for each supplier, respectively. Figure 6 reveals lowering the temperature from its current setting of 300 to 291 while raising the pressure from 25 to 29 psi moves the process mean for registration error to virtually zero. The scrap rate is 0.05%.

The big surprise is in Figure 7. Here, lowering the temperature to 281 and raising the pressure to 30 psi with the low cost epoxy leads to a zero predicted registration error and a scrap rate of 0.05%.

In the institutionalization step, the improvement team recommended switching to the low cost supplier and standardizing the optimal process settings. The $5 million a year savings resulted from lower material costs and reduced scrap.

Design for Six Sigma
It is even more cost effective to use DOE earlier in the production cycle. In design for Six Sigma (DFSS), a new business operation integrates DOE and Six Sigma from the outset. DOE becomes the analytical tool of choice for developing new products and implementing new systems. The potential savings from DFSS dwarf the alternative choice of implementing a process in the traditional way and using Six Sigma techniques to optimize it later.

Of course, DOE is not unique to Six Sigma. Invented in the early part of the 20th century, it has made contributions to virtually every area of science and technology. We recognize designed experiments require more effort than the mere collection and analysis of data. They also require some up-front investment in system time, personnel costs and material resources.

As a result, it may be tempting to rely solely on the analysis of historical data for recommended changes. It is wise to resist this temptation. The standard use of designed experiments in Six Sigma projects surely enhances their probability of immediate success. The process understanding they generate can also pay dividends far into the future.

This article is adapted from an article originally published in the February 2002 issue of Ýxtra Ordinary Sense, the official publication of the International Society of Six Sigma Professionals.


  1. W. Edwards Deming, Out of the Crisis, MIT Press, 2000.
  2. Mikel Harry and Richard Schroeder, Six Sigma, Currency, 2000.
  3. Gray McQuarrie, “Control of Key Registration Variables for Improved Process Yields on Dense MLBs,” IPC Expo ’99, March 1999, Long Beach, CA.
  4. Gray McQuarrie, “Using DOE to Solve Compensation Problems,” PC Fab, April 2001.
  5. JMP is statistical software produced by SAS Institute Inc.

Featured advertisers

ASQ is a global community of people passionate about quality, who use the tools, their ideas and expertise to make our world work better. ASQ: The Global Voice of Quality.