2019

STATISTICS ROUNDTABLE

Weighing Your Options

Decision making with the Pareto-front approach

by Christine Anderson-Cook and Lu Lu

When we are faced with choosing the best solution for a problem, most of us are very good at picking the right solution if there’s only a single quantitative objective over which to optimize.

We often get to trouble when there is no single quantitative measure to rank (Should I get the black or brown shoes?), or when there are several competing objectives to be balanced with no unique ordering of options based on multiple criteria.

The real world is rarely as simple as the first scenario: a single goal easily captured by a quantitative measure. To make choosing straightforward, often you try to pretend your problem actually fits into this category. You might focus on the most important of the attributes and not consider secondary objectives until later. Or you might make a subjective choice about how much individual objectives should contribute to the overall decision and use that choice to search for the optimal solution based on an integrated summary index. 

Choosing a supplier

Consider this: A production manager is trying to evaluate potential suppliers for her company. After considerable research and data collection, she is able to obtain the relevant information about the different alternatives. Recognizing that cost and quality are important, she seeks to make a decision that finds an appropriate balance between these two objectives.

Figure 1 shows the 14 different alternatives with the range of possible per-unit costs and quality measures. Minimizing cost while maximizing quality would be the ideal solution. From the plot, however, it is clear that no supplier simultaneously has the lowest cost and the highest quality. The supplier with the lowest cost per unit (denoted with the blue square) has relatively low quality, while the supplier with the highest quality (denoted with the green triangle) has relatively high per-unit cost.

Figure 1

The start of the decision-making process is extremely important and involves deciding on the important measures to optimize. Here, cost and quality were determined to be the key characteristics on which to base the decision.

You can imagine that quantifying quality could be done many ways, and finding one or more numerical measures that accurately characterize the key aspects of quality may require some thought and effort. It also  helps to see if there are other aspects to the decision that should be considered. For example, the timeliness of orders being filled also may have been relevant to making a good decision.

Next, you must obtain measures for all of the possible choices. Here, this resulted in the data shown in Figure 1. Clearly, if the manager had just focused on a single characteristic (such as cost), the data collection might have been easier. But there wouldn’t have been a way to assess the range of alternatives for the other measure (quality).

It’s important to realize that the absence of information on a variety of characteristics may make decision making easier and more comfortable in the short term, but it also can lead to nasty surprises in the long run.

After you have this information for all of our alternative choices, you are ready to begin the formal evaluation.The overall approach can be divided into two stages:

  1. The first objective stage constructs the Pareto front to find the set of contending solutions for further consideration. Constructing a Pareto front1 identifies the collection of contenders by eliminating inferior choices.
  2. The second subjective stage evaluates the set of choices, selects a single final solution based on the users’ priorities and carefully evaluates the potential impacts from subjective aspects of the decision-making process.
     

Narrowing the choices

The Pareto front multiple-objective optimization approach2-4 provides a strategy for examining different alternatives and guiding users to informed rational decisions, which can be tailored to their priorities. To illustrate the method, consider the example of the process for the supplier.

Finding the Pareto front, shown with the solid symbols in Figure 2, is based on eliminating the non-contending alternatives. A choice is Pareto inferior if another choice exists that is at least as good on all criteria and strictly better on at least one. For any supplier not on the Pareto front (open diamonds in Figure 2), there is at least one solution on the Pareto front that has at least as low of a cost (further to the left) and at least as high of quality (higher on the plot). Therefore, it is rational to eliminate these non-contenders from further consideration.

Figure 2

Why would anyone want to select a more expensive, lower-quality option? The only reasonable answer would be that there is another criterion we should be considering, hence the importance of identifying all of the characteristics to base the decision on at the start of the process.

In this example, assume cost and quality were determined to be the two objectives to focus on, and hence you can now focus on just those suppliers with cost and quality combinations on the Pareto front.

The set of choices on the Pareto front is an improvement over just focusing on a single criterion. Reducing the number of choices from the original 14 possible suppliers makes the comparison more manageable. Note the cheapest (blue square of Figure 1) and highest-quality suppliers (green triangle) are automatically included on the Pareto front.

In addition to these alternatives, however, you also have a range of choices that represent compromises between these extremes. For cost and quality, if you make a small compromise from the best value and select the adjacent point on the Pareto front, you can make disproportionate gains in terms of the other objective.

If you increase the cost per unit from 2 to 2.1 (a 5% increase), you’re able to achieve a 55% increase in quality (from 4 to 6.2). Similarly, by decreasing quality from 10 to 9.7 (a 3% decrease), you can obtain a 27% decrease in cost per unit (from 5.6 to 4.1). This disproportionate improvement of one objective by focusing on near-optimal instead of optimal for the other objective occurs commonly and can lead to better overall solutions.

Having all the key aspects of decision-making features identified and evaluated at the beginning is important for understanding the interrelationship and the trade-offs between these characteristics. This is helpful for reevaluating the priorities of the study and the appropriateness of the quantitative measures for capturing the main characteristics of interest.

It’s easy to pull out a lower-dimensional summary from a higher-dimensional Pareto front if some of the aspects are found to be less important in the final decision. But if some contending options have been identified based on a small dimension of objectives, and extra dimensions of decision-making factors need to be added later, the entire analysis must be repeated from the beginning. So it’s important to carefully identify the set of objectives that have potential to contribute to the decision making at the early stage.

After you have eliminated the non-competitive choices for suppliers using the objective Pareto front optimization approach, you want to further reduce your choices and eventually select a single solution. Which of the solutions (in this instance, suppliers) on the Pareto front is the right one for the decision maker is a function of how much she values cost and quality relative to each other.

Perhaps choosing the highest quality items is the top priority, or maybe the lowest cost is more important. If you’re interested in one of the compromise alternatives, a common approach for combining different criteria is the desirability function (DF) approach,5 which converts the criteria (in this example, cost and quality) into scales from 0 to 1, with the best values being mapped to 1 (in this example, lowest cost and highest quality), and the worst values being mapped to 0.

Best and worst

The choice of what value of each criterion to map to 1 is generally straightforward because there is a best possible answer based on optimizing that criterion alone. But there are different options for how to select the worst value. One option is to select the worst value found on the Pareto front. Another alternative is for the user to choose a value that feels comparable in desirability for each of the criteria.

In this case, a per-unit cost of six may seem similarly undesirable to a quality score of four. In this way, the scaled measure for each criterion may be deemed comparable. This user-specified scaling is useful when the decision maker has a good sense of how different criteria values compare in practice.

Another important subjective choice made by the decision maker is to select the format of DF for aggregating multiple desirabilities into a combined index (for example, additively or multiplicatively). This decision is relevant to how severe the penalty is for poor performance.

Depending on the selected scaling and DF format, traditionally, a single set of weights for the expression, Desirability = wcostccost + wqualitycquality, is chosen, for instance, wcost = 0.3, wquality = 0.7, and the best supplier would be chosen that maximizes the desirability score. But the Pareto front is independent of these subjective choices, making it easier to evaluate their potential imparts without requiring much extra computational effort.

Most importantly, the Pareto front approach allows the user to see the impact of uncertainty for the weights chosen for the relative emphasis to put on the different criteria. For example, what if the manager thinks wcost might be as low as 0.2 or as high as 0.4? What impact would this have on the final choice? There are several alternative summaries that can be helpful for making the decision, including a trade-off plot (Figure 3), a mixture plot (Figure 4) and synthesized efficiency plots (Figure 5).

Figure 3

Figure 4

Figure 5

The trade-off plot6 allows the user to see how different solutions compare on the [0,1] scaled criteria and raw values, as well as the number of trade-offs between the alternative solutions. The mixture plot7 identifies the best solution for different combinations of weights, wcost, wquality  in a simplex and shows the robustness of individual solutions to the uncertainty associated with the weight specification. For the choice wcost = 0.3, wquality = 0.7, the best choice is supplier 5, with supplier 6 also being best for some weights of wcost in [0.2, 0.4]. The synthesized efficiency plot8 shows how a particular solution performs relative to the best option for each weight combination.

In this case, the production manager wanted good quality as the primary criterion, but also with some cost consciousness. Based on the summaries, supplier five was deemed the best choice because it satisfied the high quality requirements (> 9) while being moderately priced (per-unit cost of 3.6).

This supplier was optimal for about 1/5 of all the desirability functions’ weight combinations (Figure 4) and had high efficiency (> 75%) relative to the best choice for a majority of all weight combinations (Figure 5). In making the final decision using the Pareto-front approach, the production manager understood the alternatives by comparing their individual criteria values, and she could make a defensible, rational decision based on a quantitative understanding of the trade-offs.

Information is power. Here, the structured understanding of the competitive choices can be empowering.


References

  1. R.T. Marler and J.S. Arora, "Survey of Multi-Objective Optimization Methods for Engineering," Structural Multidisciplinary Optimization, Vol. 26, No. 6, 2004, pp. 369-395.
  2. Lu Lu, C.M. Anderson-Cook and T.J. Robinson, "Optimization of Designed Experiments Based on Multiple Criteria Using a Pareto Frontier," Technometrics, Vol. 53, No. 4, pp. 353-365.
  3. Lu Lu and C.M. Anderson-Cook, "Rethinking the Optimal Response Surface Design for a First-Order Model With Two-Factor Interactions, When Protecting Against Curvature," Quality Engineering, Vol. 24, No. 3, 2012, pp. 404-422.
  4. Lu Lu and C.M. Anderson-Cook and T.J. Robinson, "A Case Study to Demonstrate Pareto Frontiers for Selecting a Best Response Surface Design While Simultaneously Optimizing Multiple Criteria," Applied Stochastic Models in Business and Industry, Vol. 28, No. 3, 2012, pp. 206-221.
  5. George Derringer and Ronald Suich, "Simultaneous Optimization of Several Response Variables," Journal of Quality Technology, Vol. 12, No. 3, 1980, pp. 214-219.
  6. Lu, "Optimization of Designed Experiments Based on Multiple Criteria Using a Pareto Frontier," see reference 2.
  7. Ibid.
  8. Lu, "Rethinking the Optimal Response Surface Design for a First-Order Model With Two-Factor Interactions, When Protecting Against Curvature," see reference 3.

Christine M. Anderson-Cook is a research scientist at Los Alamos National Laboratory in Los Alamos, NM. She earned a doctorate in statistics from the University of Waterloo in Ontario. Anderson-Cook is a fellow of both the American Statistical Association and ASQ.

Lu Lu is a postdoctoral research associate at Los Alamos National Laboratory. She earned a doctorate in statistics from Iowa State University in Ames, IA.


Average Rating

Rating

Out of 0 Ratings
Rate this article

Add Comments

View comments
Comments FAQ


Featured advertisers