Evaluation

Making decisions is easy when the context is familiar. Throw in some uncertainty, and choices become murky–and as we suggest in chapter 12, the processes for making decisions become less clear as well. Tools that support effective evaluation with existing products might not provide the right guidance when applied to breakthrough–and therefore untested–innovations.

In Chapter 12, we offer an array of tools and show when to use them:

  • The Decision Matrix evaluates early concepts against design principles or other attributes you think are valuable, and can help you group concepts into fast track, develop, hold, shop out, or eliminate categories.
  • Discount Cash Flow analysis and Net Present Value work with existing offerings and adjacent extensions–but Cumulative Probability and Tornado Charts are better for innovations.
  • Real Options Valuation includes ranges of uncertainties in adjusting the potential value of concepts.
  • Pilot Testing represents the smallest possible test you can make in-market, while spinning a company out as a Start-up is a bigger bet. Both reveal valuable information for the least familiar and furthest out ideas.

What seems to be working

  • The notion of an array of tools, with a way to choose when to use them, is sound.
  • Our explanation of the Decision Matrix is the most fully developed of the tools, which is good given its importance.

Areas that need more work or input

  • We know more now about how companies evaluate choices—and some of additional factors in play—e.g., beyond customer or financial value, there’s brand value, potential for creating a platform, the insights gained or capabilities developed could be leveraged elsewhere, game theory and ways of outflanking competitors. Our tools and discussion should address them, in addition to further outlining the methods to which we previously referred.
  • Although we touched on spinning out pilots or start-ups to understand value, Eric Ries’ book The Lean Startup (2011) has redefined the idea. The integrated build–experiment–adapt–rebuild cycle he outlines needs to be included as part of a portfolio of process which would compliment a portfolio of priority.
  • Many readers work in companies that already have decision making approaches—some of which don’t always align with innovation objectives. Might it be useful to suggest alternatives or ways to work within existing systems?
  • Are there other decision making issues you’ve faced, or useful approaches, in your work we should know about?
After you’ve read the chapter, please post your thoughts, constructive critique, and suggestions below. Thank you!
  • Bruno

    I see first hand everyday the damage of using simple business model or calculations for evaluation. This is maybe one of the biggest impediment to innovation in large organizations

    I like the approach and matrix for categorizing. Similar matrix have been used to categorize risks in technology projects for years.

    A couple of improvements or idea to the model you propose:
    - looking at the missed opportunity cost
    - assumptions and validating them
    - adding a time and resources component

    1- Missed opportunity Cost:
    Even once categorized in the matrix, there is most likely not enough resources to do all of them. One angle to look is not only what to do, but what it implies _not_ doing. If 3 concepts are selected, what are the concepts that will never be possible due to lack of required resources.

    2- Assumptions
    At the stage most concepts are evaluated, they are based on a set of assumptions. Those assumptions are not proven wrong or right yet. In the Develop or Hold area, the lean startup exercise goal should be to measure and quantify the assumptions to see if the concepts moves towards fast track or eliminate.

    3- Time/Resources
    At this stage, concept can be further prioritized based on the effort & time it would take to bring them to consumers. A concept could be a clear fast track, but lack of specific resources could mean that the time it hit the market makes it loose its relevance

    • http://www.creativeslant.com/ Zachary Jean Paradis

      Bruno – great feedback across the board (again). You’ve rapidly become a Naked Innovation all-star.

      We should address all of the improvements you suggest. I do see some relation to point #1 and point #2 though, as missed opportunity is often related to the assumptions you’ve made.

      I’m thinking we should have a sidebar which are potential criteria beyond what we cover now in our priority matrix, as well as a section on “other considerations to evaluation”. That section could address assumptions and validation, as well as introduce the notion of a portfolio of approach to compliment priority.

      Keep the feedback coming! We’re back from an extended holiday hiatus and now offering the entire book for free in a single file. Grab the link in the News section.

      • Bruno

        Happy to help!