The Democratization of Math

Historically speaking, why haven’t companies used statistics to manage their innovation portfolios? I’ve had this question a lot lately.

It’s a great question. Innovation is one of the most risky aspects of business, and statistics are the tools of risk management. Yet for some reason, despite 70% – 90% failure rates and trillions of dollars spent by companies each year in new product launches, acquisitions and venture capital, innovation has somehow remained a statistics-free-zone.

Why can’t we calculate the risks of innovation investments?  Why can’t probabilities help us find innovation opportunities, decide what to fund, pivot projects, shape portfolios and apportion resources?

I’m no historian, but as someone who uses statistics to manage innovation portfolios, it would have been a lot harder to do twenty years ago.

Not long ago, if your job used statistics you were probably an engineer, an economist or an actuary.  Actuaries and economists have been sitting quietly in society’s backrooms for thousands of years, crunching the numbers all of us assume “someone” is crunching. Most people associate actuaries with insurance companies, where they figure out your risk of getting sick or having a car crash before deciding what to charge you. Other actuaries sit in banks and investment firms, trying to figure out the risks of stocks, bonds, currencies and other things to help investors make money. avoiding-risk

The sad truth about statistics; they aren’t much fun.  No kid grows up wanting to do statistics for an insurance company.  Learning about probabilities and risk management is tedious and hard. It takes special training. Unlike movie stars, number crunchers also aren’t society’s most beloved members because it’s often their job to assign cold dollar figures to priceless, emotional human events (ex. death, sickness, injury). However – like plumbers, trash collectors and funeral directors – they’ve been accepted as necessary members of society who do a dirty job that, regrettably, someone has to do.

Then, twenty years ago this began to change.  In one generation, actuarial science and econometrics crept up from obscurity to reshape almost every major field of human endeavor. From roots in universities, insurance and finance, statistics now inform how doctors diagnose patients, how judges issue jail sentences, how athletes train and coaches recruit, how elections are run, how travel is booked, how romantic dates are set up, how children are taught, how college students are admitted, how lost autistic kids are found, how drugs are tested, how movies are made, how crime is deterred and how wars are fought.

The explosion in statistics marched in lockstep with the rise of modern computing. There are a few things you need before you can spread statistics into a new domain. First, you need data. You also need something to “crunch” those data with – it used to be paper, then abacus, then slide rules, then calculators and now computers.

For obvious reasons, the rise of modern computing has made every ingredient of statistics more abundant, cheap and accessible than ever before. Data is everywhere. Computing power is everywhere. Free software has never been more powerful. As a result, statistical analyses that used to be the exclusive domain of highly trained “quants” with elite resources can now be learned by teenagers with free open source software, a laptop and a little instruction from YouTube. This merger of statistics and technology gave rise to what’s now called “data science”. Math is becoming democratized.

As hard math moved from exotic to mainstream, people have become refreshingly more comfortable with the idea of using statistics to shape innovation efforts. When I first started doing this kind of work nearly a decade ago, the very premise of bringing statistics into the “creative” world of innovation struck lots of people as either offensively naïve at one extreme, or offensively arrogant at the other extreme. It was an alien concept. There was no word for “big data” or “data science.” There was no iPhone. Blockbuster was bigger than Netflix. It wasn’t that long ago, but in tech years it was ages ago.

Now words like data science and even “analytics” have entered the modern lexicon, sparing me at least half an hour of explanation in every conversation. People have stopped saying it’s impossible to use statistics in innovation. Instead they’ve started asking; “why haven’t people always used statistics to manage innovation portfolios?”. What was once offensive has now become obvious, and the change has been refreshing. Hard math is becoming easier and more accessible, allowing it to enter new domains faster than ever. This is leading to new questions, new answers and world where knowledge-seeking has become noticeably more democratized.

This Post Has 3 Comments

  1. Good read, I see statistics becoming part of everything. I am not a math person but as someone else can do the math for me, it lets us do better in what we do. My human resources team now uses statistics when hiring technicians and I think by the last two hires the results are superior than before. Don’t know if math is the reason but it is interesting to see the new approach.

  2. Nice post Thomas. There are highly innovative and risky areas like drug development for instance where advanced statistics have been applied for a long time too.

  3. Thomas, Thank you for bringing your thoughts to our attention. I too have been a strong proponent of, and written articles on, the advantages of applying analytic approaches (number crunching, statistics, data-based decision making) to managing innovation portfolios.

    As I am sure you realize, there are several challenges in simply using “statistics to manage innovation portfolios.” A more obvious one is that for most innovation concepts, there are no statistics because the concept itself is new and different. Of course, there might be suggestive trends of related products, but models and assumptions are required to bridge the differences, not statistics.

    Further, many innovation concepts cannot be solely defined by quantitative measures, but must also encompass qualitative factors such as alignment to strategy and customer appeal.

    Finally, innovation portfolios have a unique feature in that a very high percentage of the concepts (80-90%+) will be abandoned before reaching the market. So, really, an innovation portfolio must be architected with a primary investment decision feature being one of quickly terminating those concepts which are not forecasted to contribute to the bottom line. In short an innovation portfolio is managed less about statistics and more about “cents and sensibility,” to borrow a phrase.

    I am fully supportive of your idea to apply more analytical processes to managing an innovation portfolio. I believe analytic insights can provide valuable guiding decision support to what otherwise has been primarily an intuitive, gut-based portfolio management process for innovation concepts.

    Scott

Leave a Reply