BACK TO BASICS
Painting the Big Picture
by Tony Gojanovic
In today’s highly competitive global business environment, a corporation’s edge often is determined by the quality of the decisions it makes. Decisions must be based on reliable, complete information—rather than on sketchy data and gut feelings. Poor decisions no longer can be absorbed—much less tolerated—by any organization.
A popular tool for a manager to use to make smart decisions is a scorecard. A scorecard comes in many different flavors; one common type is based on color coding a point estimate—usually a mean statistic—compared to a predetermined standard. One color indicates bad or unacceptable and another indicates good or acceptable.
Table 1 is an example of a scorecard that
tracks the mean of oxygen levels in individually bottled and
packaged beer each month for a brewery.
Dissolved oxygen in beer is undesirable because it can oxidize the product and shorten shelf life. This limits the consumer’s acceptance of the beer and the distribution operation’s flexibility in delivering the product. Sometimes beer is inadvertently exposed to oxygen during packaging, but there are ways to minimize the total amount present.
In Table 1, the darker shade of gray in the “average” column indicates acceptable levels. The brewer’s goal is to have fewer than 150 parts per billion (ppb) of oxygen in each bottle of beer. This basic scorecard’s results are alarming: In nine out of the 12 months, the levels of oxygen in the beer show unacceptable readings.
Should the brewer be concerned?
Table 1 is several brush strokes away from painting a complete picture. For instance, the scorecard does not indicate variability or provide insight into what proportion of packaged beer leaves the factory containing oxygen levels above 150 ppb.
But it’s still taken seriously. Quality meetings ensue. Rather than focus on the big picture, the managers are distracted by scorecard colors that sometimes paint a different scene than what is really happening.
This example of a basic scorecard is commonplace: Based on averages, the scorecard is easy to calculate and understand but often at the expense of providing a complete picture of what’s truly happening.
In the beer example, the months from January to June are all rated unacceptable. Panicked managers developed improvement plans to cascade to the shop floor, hoping to turn the entire average column of the scorecard to the acceptable color, or, in this case, the darker shade of gray.
The improvement plans seem to work—but only temporarily. The scores for the months of July and August turn to the acceptable darker shade and managers breathe a sigh of relief. September is scored back to an unacceptable rating, and new plans are developed. The cycle continues indefinitely.
There is a more holistic and simple method to provide a comprehensive picture and understand what’s really happening in the process: a boxplot.
A boxplot is a graphical depiction of a set of data in which the box portion represents the 25th and 75th percentiles of the data. Within the box, a line indicates the median value of data—50% of the values are above and 50% are below the line.
The asterisks plotted on the graph are called outliers, or unusual points, and the lines emanating from the box represent the minimum and maximum values that are not outliers. Applying the boxplot method to the beer example makes more information and observations more apparent.
Figure 1 (p. 96) shows boxplots of monthly
oxygen data—again the goal is to send off each beer bottle
with less than 150 ppb of oxygen. The boxplots clearly indicate
the majority of product doesn’t have dissolved oxygen
levels below 150 ppb, which could not be determined by the
scorecard in Table 1 (p. 96).
Outliers indeed affected the average. The monthly averages shown in the Table 1 scorecard were unduly influenced by outliers. Specifically, the months of January through April and November are scored as unacceptable when they should have been coded as acceptable, or the darker gray. The remaining months also were much lower than the averages would indicate, even though the color codings were correct.
Because outliers also influence process capability calculations, the boxplots provide a robust approximation to capability. At the very best, the tops of the boxes never go below 150 ppb, which indicates at least 25% or more of the product produced is above 150 ppb. Since the median values in some cases are above 150 ppb, this indicates a negative capability index.
This inference is not possible from the simple scorecard analysis but is quite evident from the boxplot analysis. Not only must the average improve, but understanding and eliminating high variability must be an imperative as well.
To say a month is acceptable simply based on an average does not translate into adequate process capability. By analyzing a year of information in this scorecard format, one would be hard pressed to claim any significant improvements had been made.
Using appropriate graphics in a big picture framework can open anyone’s eyes to what is really happening within a business operation. After a simple 15 minute tutorial in reading boxplots, any manager, supervisor or line operator can analyze data within the context of his or her own experience in the process.
Fundamental questions can be asked. More importantly, decision- makers can evaluate thoughtful answers based on reliable information.
Rather than having knee-jerk reactions to color coded averages, managers can develop solutions that provide an intelligent and ethical basis for action with the big picture in mind.
- Everitt, B.S., The Cambridge Dictionary of Statistics, Cambridge University Press, 1998.
- Hoaglin, David C., Frederick Mosteller and John W. Tukey, Understanding Robust and Exploratory Data Analysis, John Wiley & Sons, 1983.
TONY GOJANOVIC is a statistician at Coors Brewing Co. and has worked in the brewery for 18 years. He obtained both bachelor’s and master’s degrees in mathematics and statistics at the University of Colorado at Boulder and Denver.