Decision Matrix

Also called: Pugh matrix, decision grid, selection matrix or grid, problem matrix, problem selection matrix, opportunity analysis, solution matrix, criteria rating form, criteria-based matrix.

A decision matrix evaluates and prioritizes a list of options. The team first establishes a list of weighted criteria and then evaluates each option against those criteria. This is a variation of the L-shaped matrix.

When to Use a Decision Matrix

  • When a list of options must be narrowed to one choice.
  • When the decision must be made on the basis of several criteria.
  • After the list of options has been reduced to a manageable number by list reduction.

Typical situations are:

  • When one improvement opportunity or problem must be selected to work on.
  • When only one solution or problem-solving approach can be implemented.
  • When only one new product can be developed.

Decision Matrix Procedure

  1. Brainstorm the evaluation criteria appropriate to the situation. If possible, involve customers in this process.
  2. Discuss and refine the list of criteria. Identify any criteria that must be included and any that must not be included. Reduce the list of criteria to those that the team believes are most important. Tools such as list reduction and multivoting may be useful here.
  3. Assign a relative weight to each criterion, based on how important that criterion is to the situation. Do this by distributing 10 points among the criteria. The assignment can be done by discussion and consensus. Or each member can assign weights, then the numbers for each criterion are added for a composite team weighting.
  4. Draw an L-shaped matrix. Write the criteria and their weights as labels along one edge and the list of options along the other edge. Usually, whichever group has fewer items occupies the vertical edge.
  5. Evaluate each choice against the criteria. There are three ways to do this:

    Method 1: Establish a rating scale for each criterion. Some options are:
    • 1, 2, 3 (1 = slight extent, 2 = some extent, 3 = great extent)
    • 1, 2, 3 (1 = low, 2 = medium, 3 = high)
    • 1, 2, 3, 4, 5 (1 = little to 5 = great)
    • 1, 4, 9 (1 = low, 4 = moderate, 9 = high)

      Make sure that your rating scales are consistent. Word your criteria and set the scales so that the high end of the scale (5 or 3) is always the rating that would tend to make you select that option: most impact on customers, greatest importance, least difficulty, greatest likelihood of success.

    Method 2: For each criterion, rank-order all options according to how well each meets the criterion. Number them with 1 being the option that is least desirable according to that criterion.

    Method 3, Pugh matrix: Establish a baseline, which may be one of the alternatives or the current product or service. For each criterion, rate each other alternative in comparison to the baseline, using scores of worse (–1), same (0), or better (+1). Finer rating scales can be used, such as 2, 1, 0, –1, –2 for a five-point scale or 3, 2, 1, 0, –1, –2, –3 for a seven-point scale. Again, be sure that positive numbers reflect desirable ratings.

  6. Multiply each option’s rating by the weight. Add the points for each option. The option with the highest score will not necessarily be the one to choose, but the relative scores can generate meaningful discussion and lead the team toward consensus

Decision Matrix Example

Figure 1 shows a decision matrix used by the customer service team at the Parisian Experience restaurant to decide which aspect of the overall problem of “long wait time” to tackle first. The problems they identified are customers waiting for the host, the waiter, the food, and the check.

The criteria they identified are “Customer pain” (how much does this negatively affect the customer?), “Ease to solve,” “Effect on other systems,” and “Speed to solve.” Originally, the criteria “Ease to solve” was written as “Difficulty to solve,” but that wording reversed the rating scale. With the current wording, a high rating on each criterion defines a state that would encourage selecting the problem: high customer pain, very easy to solve, high effect on other systems, and quick solution.

Decision Matrix Example

Figure 1 Decision Matrix Example

“Customer pain” has been weighted with 5 points, showing that the team considers it by far the most important criterion, compared to 1 or 2 points for the others.

The team chose a rating scale of high = 3, medium = 2, and low = 1. For example, let’s look at the problem “Customers wait for food.” The customer pain is medium (2), because the restaurant ambiance is nice. This problem would not be easy to solve (low ease = 1), as it involves both waiters and kitchen staff. The effect on other systems is medium (2), because waiters have to make several trips to the kitchen. The problem will take a while to solve (low speed = 1), as the kitchen is cramped and inflexible. (Notice that this has forced a guess about the ultimate solution: kitchen redesign. This may or may not be a good guess.)

Each rating is multiplied by the weight for that criterion. For example, “Customer pain” (weight of 5) for “Customers wait for host” rates high (3) for a score of 15. The scores are added across the rows to obtain a total for each problem. “Customers wait for host” has the highest score at 28. Since the next highest score is 18, the host problem probably should be addressed first.

Decision Matrix Considerations

  • A very long list of options can first be shortened with a tool such as list reduction or multivoting.
  • Criteria that are often used fall under the general categories of effectiveness, feasibility, capability, cost, time required, support or enthusiasm (of team and of others). Here are other commonly used criteria:
    For selecting a problem or an improvement opportunity:
    • Within control of the team
    • Financial payback
    • Resources required (for example; money and people)
    • Customer pain caused by the problem
    • Urgency of problem
    • Team interest or buy-in U
    • Effect on other systems
    • Management interest or support
    • Difficulty of solving
    • Time required to solve.
    For selecting a solution:
    • Root causes addressed by this solution
    • Extent of resolution of problem
    • Cost to implement (for example, money and time
    • Return on investment; availability of resources (people, time)
    • Ease of implementation
    • Time until solution is fully implemented
    • Cost to maintain (for example, money and time)
    • Ease of maintenance
    • Support or opposition to the solution
    • Enthusiasm by team members
    • Team control of the solution
    • Safety, health, or environmental factors
    • Training factors
    • Potential effects on other systems
    • Potential effects on customers or suppliers
    • Value to customer
    • Potential problems during implementation
    • Potential negative consequences.
  • This matrix can be used to compare opinions. When possible, however, it is better used to summarize data that have been collected about the various criteria.
  • Sub-teams can be formed to collect data on the various criteria.
  • Several criteria for selecting a problem or improvement opportunity require guesses about the ultimate solution. For example: evaluating resources required, payback, difficulty to solve, and time required to solve. Therefore, your rating of the options will be only as good as your assumptions about the solutions.
  • It’s critical that the high end of the criteria scale (5 or 3) always is the end you would want to choose. Criteria such as cost, resource use and difficulty can cause mix-ups: low cost is highly desirable! If your rating scale sometimes rates a desirable state as 5 and sometimes as 1, you will not get correct results. You can avoid this by rewording your criteria: Say “low cost” instead of “cost”; “ease” instead of “difficulty.” Or, in the matrix column headings, write what generates low and high ratings. For example:

    Importance

    Cost

    Difficulty

    low = 1 high = 5

    high = 1 low = 5

    high = 1 low = 5

  • When evaluating options by method 1, some people prefer to think about just one option, rating each criterion in turn across the whole matrix, and then doing the next option and so on. Others prefer to think about one criterion, working down the matrix for all options, then going on to the next criterion. Take your pick.
  • If individuals on the team assign different ratings to the same criterion, discuss this so people can learn from each other’s views and arrive at a consensus. Do not average the ratings or vote for the most popular one.
  • In some versions of this tool, the sum of the unweighted scores is also calculated and both totals are studied for guidance toward a decision.
  • When this tool is used to choose a plan, solution, or new product, results can be used to improve options. An option that ranks highly overall but has low scores on criteria A and B can be modified with ideas from options that score well on A and B. This combining and improving can be done for every option, and then the decision matrix used again to evaluate the new options.

Excerpted from Nancy R. Tague’s The Quality Toolbox, Second Edition, ASQ Quality Press, 2004, pages 219–223.

Featured advertisers


ASQ is a global community of people passionate about quality, who use the tools, their ideas and expertise to make our world work better. ASQ: The Global Voice of Quality.
Quality Tools