Has your innovation guru been tested? The answer might surprise you

Would you make loved ones fly in an airplane that hadn’t been safety tested? Would you give your child a drug that skipped out on clinical trials? Probably not (at least I hope not). How about your innovation? Would you bet your career, salary and colleagues on advice that had never been tested? No… of course not… well… actually… yes.

Name one innovation guru who has subjected his/her pet theory to predictive testing.

Go on.

I’ll wait.

Think of the last innovation book you read. When was the last time you saw numbered “steps to success” from a guru? Write the examples down, then go back and see if any were tested. Did they meet the minimum requirements of the Scientific Method most of us learned in science class? The purpose of the Scientific Method isn’t to be uptight or formalistic. It’s the very process of separating good ideas from bad. Fact from fiction. Truth from falsehood. Reality from illusion.

With only a few exceptions – who probably aren’t the ones you’re thinking of – innovation gurus don’t subject their theories to predictive testing or validation. It simply isn’t done and, until now, you probably hadn’t noticed.

How can this be? How is it possible that something like 99.991% of the innovation advice you’ve enjoyed has endured zero predictive vetting whatsoever? The answer is: it probably never occurred to the guru either.

Guru2

It isn’t that your favorite gurus are necessarily dishonest, lazy, sloppy or ignorant. Chances are they’re extremely intelligent, well intentioned and have a lot of insight. Yet once they come up with a theory, for some reason the discipline of “innovation” has never asked them to take the next obvious step – to test the theory going forward. It simply hasn’t been part of the discipline.

Innovation gurus more or less follow this process when building theories:

  1. Look at past data. Whether it’s a couple first-hand examples or a huge pile of historical data, gurus begin by examining history.
  2. Notice patterns. Most of the time historical data is next whittled down to a manageable number of cases (from three to a few dozen examples). These are usually examples of companies that have done extremely well in one way or another. From this smaller number, gurus next try to find any patterns shared by the examples.
  3. Brand the patterns and prescribe them to the world. Write a book. Speak at TED. Teach MBAs. Sit on boards. Sell consulting. Have your picture taken with the President. Dish with Oprah. Do your best to sound scientific and inspirational the whole time.

What’s missing are critical next steps: predictions from the hypothesis, experiments, randomized testing, statistical vetting. It’s one thing to find commonalities between a small number of hand-picked examples, and another to test if those traits have any predictive value in the real world beyond those limited cases.

ideation cartoon Consider this the next time you pay top dollar for a sexy, groovy, hosted ideation workshop. You walk out feeling, down to your bones, that your ideas have been improved. Yet, if you think about it, you have absolutely no idea if those ideas are any more likely to succeed than they were last week. If you then ask the guru how the session specifically changed your statistical odds of success, you can imagine the blank stare followed by a flurry of handwaving. I’m not picking on ideation workshops, it was just an example. The point isn’t to say any specific innovation theory is good or bad – just that we don’t know. Nobody bothers to test and measure.

The problem is simple – it’s relatively easy to design a feel-good brainstorm or find commonalities between just about anything. For example, what do Mozart, Theodore Roosevelt and Thomas Edison have in common? Given a few hours, most of us could come up with a lot of “patterns” shared by these men. Given a year we could also write a book about how the patterns can make any innovation more successful. We could add tons of detail about each person, sprinkling in novel and delightful facts, anecdotes and tidbits to solidify our points. What’s wrong with that?

I’ll tell you what’s wrong. For starters, it’s unsubstantiated hogwash. We just pulled it out of our… ears. As such, our advice is just as likely to do harm as it is to do good.

It isn’t a victimless crime. Whenever we rely on un-tested advice during major business decisions, we’re putting the fate of our innovations at risk. Bad advice can kill good businesses. There’s a word for this in medicine, law and many other advice-giving fields: malpractice.

If you’re sick of living in a world where 70% – 80% of new innovations fail, it’s time to stop condoning malpractice. At the very least, it begs for all of us – consumers of innovation theory – to become tougher critics and to demand more from gurus. Otherwise we’re just feeding, and funding, rampant malpractice in a domain society depends on for its very future.

What about you? Let’s be honest. Has your favorite innovation guru been tested? The answer might surprise you.

This Post Has 8 Comments

  1. Gurus do not like to test theories because it’s too much work and they may find out they are wrong at the end. It is easier to get rich as a famous consultant if you keep appeal to emotion and star power instead of grounded in reality. Why do the extra work if it can interfere with a guru success and nobody is asking hard questions?
    Yes this needs to change, I agree. But the change will not come from gurus. They are happier the way things are.

  2. You can’t test something that is intangible like innovation. Creativity can’t be measured.

  3. A agree with that. As a matter of fact, I am living in an industry that has been using the same technology for a century. (There are specific historic reasons.)
    The result is that development is 100% step by step, 100% done by tests (practice).
    The result being that nobody ever cared to check if that practice was respecting NEWTON’s laws anymore. “Practice is where everything works but nobody knows why.”
    All this to say that because technology is a century old and not challenged anymore:
    It has been replaced by various dogmas and magic formulas.
    And our GURUS (sometimes chief engineers of major companies) are not afraid of stating so in the medias.
    For instance stating that they are using the “Magic triangle dogma”.
    Dogma which has got no possible rational explanation. As a matter of fact if you study this “magic triangle”, you soon find out it’s a joke.
    So GURUS are everywhere, not only in the innovation microcosm…

  4. … and for GURUS: scientific dialogue is not an option.
    So, they commit dogmas that they are offering as “true theories”. lol

  5. Love to test the Gurus. What tests do we suggest?

    While it is always good to be critical, Innovation often takes a long time and the actual data to Theory Model would be at best like a social experiment correlation. It is unlikely to be a 95% correlation like a design of experiment transfer function. More like Quantum Mechanics multiple states and fuzzy probability theory.
    Innovation Theory is still relatively new and still emerging into a more rigorous methodology. Not a science yet in my opinion. Neither is Marketing on Front End.

    But love the idea and I think we will get there.
    Bill

  6. Hi Thomas, I appreciate very much the posting, it is true that many of the so called gurus in Innovation are gurus because they promote themselves but if you look at their track record they have no expertise in bringing something to market.

    I read many articles by people having zero operational expertise in something, thinking that just because they suggest to place an app they became true innovators. There are others that advocate that Innovation is the exclusive realm of the R&D people. In both cases they are wrong, incremental innovation has as much value as radical innovation.

    As the article states there is a need to quantify the benefits of something. Most of the companies pay lip service to Innovation and will never be able to reap its true benefits, because it is not sufficient hiring a self proclaimed guru to transform the organization overnight. It requires dedicated upper management and motivated rank and file. It also requires the allocation of time and money.

  7. You can’t always quantify the inputs of an innovation, such as all the things that went into the idea and the process of commercialization, but you can usually quantify the outputs, such as if the business succeeded or not. We need to start by quantifying outputs and then begin matching them with inputs to see what worked and what did not. This can and should be done by companies given all the money at stake. Companies will spend millions on an idea, but not a dime on how effective their innovation process is overall. This needs to change.

  8. I guess it actually depends on what the person is expert of. If he is more of the “Skills Guru”, then he might have already tried and tested it.

Leave a Reply