Weighted Decision Matrix

Where the decision matrix came from

Many websites call any weighted decision matrix a "Pugh matrix." The two are related, but they are not the same thing. Stuart Pugh, a British professor of engineering design, taught a method for choosing between early product designs and set it out in his 1991 book Total Design¹. In his version, you pick one design as the reference point. Then you mark every other design as better (+), worse (-) or the same (S) as the reference on each criterion. You count the marks, improve the weak designs, and run the comparison again. It was meant to be repeated, not to produce a single final score.

The weighted version this tool builds comes from a different line of work: multi-criteria decision analysis, the study of choosing when several goals pull in different directions. Ralph Keeney and Howard Raiffa's Decisions with Multiple Objectives, first published in 1976, is the standard text². Its simplest model is the one in the table above: give each criterion a weight, score each option, multiply, and add.

Knowing this helps in one way. Adding up weighted scores assumes a high score on one criterion can make up for a low score on another. That is often fine. When it is not, you need a different step, covered below.

How to build a decision matrix well

Set the criteria and weights before you look closely at the options. Once people have a favorite, they tend to pick the criteria and weights that make it win, often without noticing. Daniel Kahneman, Dan Lovallo and Olivier Sibony make this the first step of their method for strategic decisions, which they built on the research into structured job interviews: decide what you will judge, and how, before judging³.

Make each criterion something you can check. "Ease of use" invites a guess. "Dispatchers booked a test job without help during the trial" can be observed.

Write down what a 1, 3 and 5 mean. Without anchors, one person's 4 is another's 2, and the total mixes them. For cost, a 5 might be "under $10,000 over three years" and a 1 "over $30,000."

Take out overlaps. "Price" and "total cost" measure the same thing. Listing both quietly doubles its weight. Four to seven criteria that do not overlap is usually enough.

Turn deal-breakers into filters, not weights. If an option must keep customer data in the country, an option that fails that test is out, whatever else it scores. Left in as a weight, strong scores elsewhere can buy it back.

A worked example, and why close totals are ties

This example is invented for illustration. A 40-person plumbing company is choosing scheduling software. The options are three made-up products.

Beta wins by 3 points out of a possible 70. Now suppose the owner says cost barely matters this year, and its weight drops from 3 to 1. Alpha scores 49 and Beta 48. One weight change flipped the result.

That is the most useful check you can run on any matrix. Change each weight by one point and see whether the winner changes. If it does, the matrix has not decided anything. It has shown you the real question, which here is "how much does cost matter to us this year?" Settle that out loud, then decide.

Common mistakes

• Adjusting weights after seeing the totals. If the answer feels wrong, say so and discuss why. Nudging a weight until the "right" option wins turns the matrix into a cover story.
• Treating scores as facts. Every number in the table is a judgment. Writing it as a number makes it look more certain than it is.
• Reading a small gap as a clear win. A difference smaller than one weight change can undo is a tie.
• Scoring only the options on the table. If every option scores poorly, the answer may be an option nobody has listed yet.

What a decision matrix does not answer

• How risky each option is. A score of 4 for a proven product and a 4 for an untested one look the same. For the options still in the running, list what could go wrong with a risk register.
• Which of many projects to do first. For sorting a long list of work by importance and urgency, a priority matrix fits better.
• Whether the choice fits the strategy. The criteria only reflect the strategy if someone put it there. For how planning methods compare on making choices, see the framework comparison.
• Who carries it out. A decision with no owner stays a decision. Once chosen, set out who does what with a RACI matrix.

Using it with a leadership team

How the conversation is run matters more than the spreadsheet. In a McKinsey study of 1,048 major business decisions, reported by Lovallo and Sibony in 2010, the quality of the discussion mattered six times more than the amount of analysis⁴.

• Agree the criteria and weights first, in a separate step, before anyone argues for an option.
• Score alone, then compare. Each person fills in the matrix on their own before the meeting. Kahneman, Sibony and Cass Sunstein call this kind of step "decision hygiene": gathering independent judgments before discussion, so early speakers do not set everyone else's view⁵.
• Spend the meeting on the biggest gaps. Where one person scored a 5 and another a 2, they usually know different facts. That is where the useful talk is.
• Name who decides. The matrix informs the choice. One named person, or the team by a stated rule, makes it, with an owner and a date for what happens next.

For bigger calls made at a team retreat, see the guide to running a leadership offsite.