Intuitive innovation – hurts so good

Oh, that thing we call “intuition.” Some call it “common sense.” Others call it their “gut,” “hunch,” or “Spidey sense.”

Intuition is one of humanity’s most celebrated notions. It’s the essence of art. Intuition is even held out as a virtue. It’s a Hollywood cliché when data-types are ignored (or punched in the face) by intuitive, action-types who end up saving the day by scoffing at the facts because it’s all about guts. Damn the torpedoes. Never tell me the odds. Intuition is why Kirk is boss and Spock will never really be more than #2. If you say anything bad about intuition you should expect some angry emails.

We know this. We feel this. Yet we also know our intuitions are pretty unreliable a lot of the time. Our feelings are just that, feelings. Feelings can be wrong, even when they feel right. Psychologists call this “cognitive bias” and there’s a ton of great work on how intuition can lead us astray. To name a just few recent books on this topic:

  1. Thinking Fast and Slow
  2. Predictably Irrational
  3. Sway
  4. Think Twice
  5. Why We Make Mistakes
  6. Mistakes Were Made (But Not By Me)
  7. Why Plans Fail: Cognitive Bias, Decision Making, and Your Business
  8. On Being Certain: Believing You Are Right Even When You’re Not
  9. Everything Is Obvious: How Common Sense Fails Us
  10. The Invisible Gorilla: How Our Intuitions Deceive Us

So… on the one hand we know our intuitions can be dead wrong a lot of the time, but on the other hand we can feel compelled to defend our intuitions to the bitter ends. Intuition is going strong. Even in a time of more data than ever, here’s how the word “intuition” has been trending in literature since 1800:

intuition trendIntuition can be especially problematic in the field of innovation. The word “innovation,” like intuition, is also nearly synonymous with art. Innovation is seen as a creative process. A melding and reshaping of the universe to create something awesome and new. No wonder the word “innovation” is so over-used, just thinking about it can feel euphoric.

The word “invention” had its heyday, but now it’s all about “innovation.” Which word feels better to you? Try saying them both to yourself. Invention. Innovation.

Invention seems so-so. Not bad, but kind of dry and uninspired. It’s as if any moron can “invent” something. Innovation, on the other hand, feels uplifting and adventurous. It makes you feel you’re changing the world – even if you’re making videogames about throwing birds at pigs. Take a gander at how invention and innovation have trended:

invention vs innovation trendWe love intuition. We love innovation. We especially love using our intuitions to innovate. Can you blame us?

Meanwhile around 70% – 80% of new innovations fail within their first 10 years. The sad empirical truth about intuitive innovation is – it hasn’t been working very well. We all know this at some level, but we still don’t want to change. We’re intuition addicts. We know Spock is right, but it’s more fun to follow Kirk into the asteroid belt.

A lot of fields have migrated away from intuition towards more scientific disciplines, and they’re better for it. Kind of a buzzkill, but this process has solved many of humanity’s toughest riddles. For example, medicine used to be almost entirely art (you’ve been cursed by an evil troll) and has now become more scientific (your blood-work stats say you have an 86% chance of infection). The same is true for engineering and the physical sciences. Why not innovation?

Some people have a tough time with this idea because it isn’t clear where to start. Sure, we’d all like innovation to be more scientific, at least in principle, but what would that even look like?  More stage-gates and milestones followed by lean startup business model canvassing that’s disruptive and crosses the chasm in an accelerator?  Those things could be fine, but that isn’t what I mean. Rather, making innovation more scientific begins with asking different kinds of questions and a willingness to start measuring stuff.  For example, I really enjoyed Moore’s Crossing the Chasm. It feels right. However a more scientific approach would go something like this:

  1. Is there a testable hypothesis in the book?
  2. Has that hypothesis been, in fact, tested in a predictive manner?
  3. If so, what percentage of the time is the hypothesis correct? How often, and in what circumstances, is it incorrect?
  4. Is there any statistical significance in its predictions or are they statistically random?
  5. What was the sample size? Was it big enough to create meaningful statistical confidence?

See the difference? Dry as it may seem, this is pretty important stuff to know before you risk everything using pointers from a bestselling innovation book.  Maybe the book’s advice is sound, maybe it isn’t. Doesn’t it bother you that you don’t really know? Think about – literally – every strategy or innovation theory you’ve ever bought into. Ever seen relatively basic scientific questions like these asked and answered?  I didn’t think so. If you think it’s in there, check again. With the billions (or trillions) spent on innovation, innovation books, innovation consulting and innovation theory each year, doesn’t the world deserve better? With all the lives, businesses and communities at stake, stop taking people’s word for it (even if they have amazing resumes) and stop following ideas just because they feel good. Demand better science.

Please consider this the next time you’re thinking about blue oceans, level 5 leadership, innovating close to the core, practicing judo strategy, design-driven innovation, business model innovation, going with your core competence, visionary leadership, first-mover advantage, fast-follower advantage, razor/razor-blade strategy, bottoms up innovation, tops down innovation, team-centric innovation, lead-user innovation, or whatever else you’re pondering.

The point isn’t to say innovation should only be about either science or art. They aren’t mutually exclusive. There’s room for both. I’ll say it again – both art and science have their places in innovation. Of course it’s a balancing act.  Yet with a 70% – 80% mortality rates for new businesses, it’s time to slide innovation a few ticks further in the direction of science. It isn’t time for wholesale abandonment of intuition, but for some deliberate recalibration. Too much intuition is hurting innovation, even while it feels so good.

This Post Has 15 Comments

  1. Victoria

    Interesting…..I am pondering what happens if you substitute the concepts of useful or helpful for science. What about intuition balanced by creating something useful or helpful to the world….?

  2. Terry Davis

    Well said Thomas! You’ve hit the nail on the head. I believe your methodology will be the norm one of these days – the sooner the better! If businesses would just step back and ask WHY they have such a dismal success rate instead of just accepting it as the norm, and apply your stated principles, our economy would prosper as never before.


  3. Krzysztof

    Very good article. And I absolutely agree: Data not intuition is what matters.
    At least for me 🙂

  4. Silverio

    La Innovación Intuitiva tiene sentido principalmente en empresas de servicios donde la producción y el consumo del servicio es simultáneo y el cliente participa de forma activa en la creación del servicio. Consiste en crear en cada caso soluciones concretas para satisfacer las necesidades del cliente, sin que exista una planificación previa. Tiene un componente sistemático (es decir, debe hacerse sistemáticamente en cualquier situación y con cualquier cliente), pero se basa en las personas y su capacidad de interacción.

    (Google Translate provided by the Editor – apologies for anything lost in translation:
    “Intuitive Innovation makes sense primarily in business services where production and consumption is simultaneous service and client actively participates in the creation of the service. Is to create in each case specific solutions to meet customer needs, without any prior planning. It has a systematic component (ie, must be systematically in any situation and with any client) but is based on people and their ability to interact.”)

  5. Eric Kennedy

    So what happened to good old fashioned common sense? If you can’t see it it isn’t there. How did the cats eye get its name.

  6. Dane

    Really? Not so fast, please. In a world of big corporate, vanilla everything because everybody’s so afraid of risk…do we really want to take away the possibility of a step-changes in innovation?

    How often do you create something new and you don’t have data? Answer: MOST OF THE TIME. That’s because it’s new. If we follow a risk/fear/data based model for innovation as Thomas suggests, there never will be any big innovations that are really new.

    What is described here is management, not innovation.

    Management is about rules and data. Innovation is about breaking rules and taking risks.

  7. Erik

    Our intuition and intuitive thoughts do not come to us magically. Personal intuition is rooted in our past personal experience, knowledge, education, etc. Humans learn through experience. That “gut” feeling is in reality based on our assumed/presumed future “effect” for a given cause or circumstance based on our past environment/experience. With that said, intuition can indeed at times serve us well from a risk management perspective.

    Of course past performance is NOT always a reliable indicator of future performance and our subjective personal assessment of past experience may not always be in reality 100% accurate. Hence “intuition” can often fail us.

    Innovation is seeing past what is currently available in the marketplace. Successful innovations are disrupters to the way society currently does things. However, successful innovation is rarely a pie in the sky idea which creates a completely new “need state” for society. Successful innovation is almost always rooted in addressing customer needs and customer pain points which already exist. Innovation is about eliminating “pain points”, creating greater efficiencies, and addressing a need differently or better.

    Many have heard the famous quote attributed to Henry Ford: “…If I had asked people what they wanted, they would have said faster horses…” Henry Ford didn’t invent the automobile, he mass produced it and made it affordable to the masses. He didn’t create the need for transportation. The need already existed. He made transportation for efficient.

    We have seen similar innovative progressions with personal entertainment (specifically music) over the past several decades. Records > Cassettes > CDs > iPod/Digital. These innovations addressed the same root “need” but did so in a “better” and more efficient way.

    Innovators with a “build it and someone will buy it” mentality will fail more often than not. Innovation should not start with a product or service designed by an inventor/innovator who then tries to find customers who value the product/service enough to part with their money. Innovation should start with a look at society. Innovation should start with a deep analytical investigation of potential customers in a market segment with the goal of identifying true customer needs and pain points which are not currently being addressed in the market place. Once the need has been established and verified the innovation of the solution can begin.

    Design and create a product or service to fill a known and verified need, not the other way around.

  8. Jericho

    Some people seem to think a more scientific approach to innovation inherently precludes the ability to do something big and new. The assumption seems to be that making innovation more scientific will make every innovation dull and incremental. I’d argue that assumption isn’t necessarily true.

    It’s possible to be more scientific, AND find new big innovations that are very different from the past. Compare it with any other science. In biology, nobody knew about germs, microbes, bacteria, etc. until science discovered it. Then the world of biology was opened to tons of big, new discoveries and was still a better discipline. This is what I hear the author saying – become more scientific in innovation, and you can still find big new ideas, or even huge new realms of possibility nobody ever knew existed.

  9. sachin kundu

    Umm, well you do mention lean startup, which is based on identifying hypothesis and validating through falsifiable experiments.

    Very scientific I would say.

    Intuition is important. Even in innovation. Intuition however needs to be verified with experimentation in the real world. If you just look for data, you will always lag behind. Data comes from previous experiences and extrapolating future is hard…very hard

  10. Shubham

    Very interesting. Thanks for sharing Thomas

  11. Eric

    Intuition is where true innovation always starts and it takes sometimes decades to find out whether or nor it was right…

  12. John

    I think you’re fundamentally right … intuition needs to give way to science wherever the problem, objective, or step in the innovation process is well defined. Cognitive bias is rampant (and I’m as guilty as anyone!) But there are a couple of anti-science components of innovation.

    One of them is “the intelligent adversary” … if the science of innovation is sufficiently developed, then your competitor can use it to derive the same result. We would then have to invent a new science of innovation in order to escape the competition.

    Another is “imperfect knowledge”. Many decisions need to be made in the absence of perfect knowledge of the situation; a purely scientific algorithm might be rendered useless if there is not sufficient inputs to derive a valid output. Here experience and intuition are often useful.

    A third is what might be called a “wholistic” or “apples and oranges” scenario, where many different and mutually unrelated (but nevertheless critical) considerations need to be balanced in selecting a strategy. It’s rare that a scientific approach correctly values all of the separate considerations in relationship to the scenario.

    I think it’s also unfair to say that intuition is always faulty. It’s pretty clear that some people in some situations have better intuition than others; perhaps their success rate rises to 50%, which is not bad. The trick is to know whose intuition works in which situations … and when scientific approaches will or won’t increase the chances of success.

  13. Thierry

    I thought I was a scientist at heart (oh, maybe this is an oxymoron!). Well, I am sad to realize I did not ask myself this kind of question; I didn’t show this level of requirement when assessing for myself the value of something like “Crossing the Chasm”. So, yes, this issue hurts!

    It is even more painful when I consider the depth and width of the gulf that remains to be taken. Do you have any idea of the effort to undertake to answer your questions on all the numerous books, essays, whatever has been said about innovation? And moreover, to do it on any single business plan or strategy or innovative product we try to shape? It is discouraging.

    Then I think… Isn’t intuition a form of science? Intuition may be the result of a kind of very fast and very human big data processing. Intuition takes into account a large number of data, lots of almost unremarkable details, signs gathered throughout a life time, a journey, or a bunch of meetings and encounters. The ability to remark these trivial facts that can help taking the right decision, and to recognize them as non-unsignificant, is not something easy to explain and to make explicit. It is not either something that can be easily taught and learned. And the list of said data, the exhaustive list of facts that were processed, can generally not be provided.

    Does it make them irrelevant? Does it make a chimera of intuition? Eventually, it is the processing algorithm itself that is certainly not a writable algorithm, not with symbols used by mathematics and computers today at least. Processing nonetheless. But mathematics is not science. The object of science is reproducible events that can be observed, described and predicted (mathematics is a tool to do so). So the question really is: is intuition reproducible? And the point is that when it is about the main claim of a book, yes it must be measurably reproducible. But when it is about my one single did as an entrepreneur, I need no second time. I need a miracle, which is by definition a non reproducible fact. Possibly, a fact nonetheless.

    I have doubts yet what I need as an innovation advisor? Should my intuitions be measurable and reproducible? Or can I consider any piece of advisory as a potential miracle doer? Art?

  14. Mike

    It depends upon what type of innovation we are discussing – sustaining or disruptive. Christensen has said that disruptive innovation is impossible to measure with traditional tools. “Failure is frequently rooted in the forced use of habitual, but inappropriate processes for doing market research, strategic planning, and budgeting.” He cites Sony under Akio Morita (1950 – 1980) as an example of history’s most successful disruptor – radios, television, VCR’s and the Walkman. Morita believed that if a market didn’t exist, then its future impact could not be measured. So he relied upon his INTUITION!

  15. Eric

    When we have not personally discovered anything really “outside the box”, and by that I mean a different way of thinking things in our field (A real jump in a science or a technology): We have NO CHANCE to have experienced intuition.

    We have to be humble enough to listen to those who have experience intuition:
    “It’s a mistake to think necessarily false, something we don’t understand…”

    And listen to what Albert has to say about it:
    “The intuitive mind is a sacred gift and the rational mind is a faithful servant.
    We have created a society that honors the servant and has forgotten the gift.”

Leave a Reply