OK, so now your team has come up with a bunch of ideas. How do they pick what to go with and develop further?
They could decide to go with the most colorful option. But they shouldn't make their decision on appearance alone – factors such as creativity, use of technology, and unexpectedness might need to be taken into consideration. So how can you make sure your team makes the best decision, while taking all of these different factors that the team values in its solution, as well as how the problem is being scored, into account?
Decision Matrix Analysis
Decision Matrix Analysis is a useful technique to use for making a decision. It's particularly powerful where you have a number of good alternatives to choose from, and many different factors to take into account. This makes it a great technique to use where there isn't a clear and obvious preferred option.
Being able to use Decision Matrix Analysis means that your team can take decisions confidently and rationally, at a time when other teams might be struggling to make a decision, or settling for the first popular idea the team comes up with.
Decision Matrix Analysis helps your team make a good decision when they need to weigh up many difficult-to-compare factors. To use this approach, use the following steps:
Step 1. List all of the options your team comes up with as rows in a table.
Step 2. List the factors the team values in their solution as column headings.
For example, if you're trying to choose between three devices, you might want to compare each one in terms of their creativity, wow factor, and reliability.
Step 3. Figure out the relative importance to the team of each factor.
Score each factor from, 0 to 5, where 0 means that the factor is absolutely unimportant in the team’s final decision, and 5 means that it is very important. (It's perfectly acceptable to have factors with the same importance.)
For example, is creativity more important than a “Wow” factor? Is reliability what matters most? What are the criteria being used to score in the problem? Risk-taking or effectiveness? These relative importance values may be obvious to the team. If they are not, then use a technique such as Paired Comparison Analysis to estimate them.
Step 4. Score each option from 0 (poor) to 5 (very well) on how well it addresses the teams chosen importance factors. Note that you do not have to have a different score for each option; if none of them are good for a particular factor in the team’s decision, then all options should score 0.
For instance, in our example, you'd give each device a score between zero and five for how well they meet the creativity, “wow”, and reliability factors.
Step 5. Multiply the scores for how well each option addresses the factor (Step 4) by the relative importance scores you’ve already entered for each factor in Step 3. Then total them up for each option.
When you've completed this last step, the option with the highest score will be the team’s best choice. Team members should then sense check this against their intuition. If some options score much higher or lower than the team expects, consider why this is, and reflect on the scores and weightings that the team applied. This may be a sign that certain factors are more important to the team than they initially thought, and a revised score or weighting is needed. Also, if an option scores very poorly for a factor, decide whether this rules it out altogether and a potential solution.
Finally, there may be two solutions with similar scores, but maybe on different factors. Is there a way to combine aspects of those solutions so as to get the best of both worlds? Or maybe you need to come up with some additional factors to evaluate these potential solutions with.
So in this example, having a "Wow Factor" was rated as much more important than reliability, leading to the Rube Goldberg device being the best scoring option. If risk taking was given more points than effectiveness then this might be a good solution. If not, the team might want to reassess the criteria to emphasize the importance of reliability. Given that a pneumatic system, and an Arduino device were much more reliable, could these technologies help improve the success of a Rube Goldberg solution?
To give it a go download our worksheet.
Paired Comparison Analysis
When your team is choosing between many different options, how do they decide on the best way forward?
This is especially challenging if their choices are quite different from one another, if decision criteria are subjective, or if the team doesn't have objective data to use for its decision.
Paired Comparison Analysis helps you to work out the relative importance of a number of different options – the classical case of "comparing apples with oranges."
There are six simple steps to follow, and you can use our downloadable worksheet to make the process even easier. This procedure can be done individually by each team member, or done as a group activity. Here's what to do.
Step 1: Make a list of all of the options that you want to compare.
Assign each option a letter (A, B, C, D, and so on) and note this down.
Step 2: Mark your options as both the row and column headings on the worksheet.
This is so that you can compare options with one-another.
Note: On the table, the cells where you will compare an option with itself are blocked out. The cells on the table where you would be duplicating a comparison are also blocked out. This ensures that you make each comparison only once.
Step 3: Within each of the blank cells, compare the option in the row with the option in the column.
Decide which of the two options is most important, and write down the letter of the most important option in the cell.
Step 4: Score the difference in importance between the options, from zero (no difference/same importance) to three (major difference/one option much more important than the other.)
Step 5: Consolidate the results by adding up the values for each of the options.
You may want to convert these values into a percentage of the total score.
Step 6: The team should use its common sense, and manually adjust the results if necessary
Here the preferred choice is Under the Sea, but Game Show had some value. Maybe some aspects of a game show, or the reasons that Game Show was liked, could be incorporated into an underwater theme?
Paired Comparison Analysis is useful for weighing up the relative importance of different options. It's particularly helpful where priorities aren't clear, where the options are completely different, where evaluation criteria are subjective, or where they're competing in importance.
The tool also provides a method for working out the relevant importance between factors when used with a Decision Matrix Analysis.
So here, the team values a "Wow Factor" much more than Reliability. A high risk, high reward choice. Maybe the team may need to revisit this in the light of testing, or this truly is the guiding principle in their chosen solution.