Brains aren’t just for Zombies. They’re for people too, although, sometimes, it seems that some people forge this. 😉

But, anyway, I have to applaud the HBR Blog Network for this recent post on Why Data Will Never Replace Thinking because it’s not all about big data. There’s a reason that we have been arguing for centuries about whether deduction from first principles or induction from observed reality is the best way to get at truth.

I tend to side with Popper’s synthesis in that the only scientific approach is to formulate hypotheses that are falsifiable. Sure, with big data, you can look at information in real time, and you can make minute adjustments, and you can build a closed-loop system, where you continuously change and adjust but I do not believe that you make no mistakes, because you’re picking up signals all the time because you never get all the signals. And even if you captured every monetary transaction, you still wouldn’t be capturing the drivers behind every transaction, which are fundamentally human in nature, and often emotional, and not captured.

As the article says, the element of hypothesis/prediction remains important, not just to science but to the pursuit of knowledge in general. We humans are quite capable of coming up with stories to explain just about anything after the fact but it’s by trying to come up with our stories beforehand, then testing them, that we can reliably learn the lessons of our experiences -— and our data. No matter how big the data gets, we still need hypothesis, and the more data, the more important the hypothesis gets — otherwise, what is all the data for?

And the quote for Nate Silver is great: data-driven predictions can succeed -— and they can fail. It is when we deny our role in the process that the odds of failure rise. Before we demand more of our data, we need to demand more of ourselves.