… And Keep Your Big Platforms. Big Brains Will Still Win in the End!

Five years ago, about the time when the big data hype first reached insane hype levels, SI published a post that it was sick of all the big data hype and how it is not the answer to all our problems because, not only is this a load of baloney, the reality is that there’s no such thing as big data in business. As we said then, relative to our ability to process it, data has always been big. And, in business, big has always been meaningless. Furthermore, in business, we’ve always been able to process as much data as we need to in reasonable amounts of time if* we make good algorithm and technology decisions

Plus, the fact that all of the hype around big data is often centered around the fact that we will be able to replace science with math and processes with AI programs is even more ridiculous. There is no such thing as artificial intelligence. And even though we’ve finally taken automated reasoning to the point that we have assisted intelligence, there’s a big difference between recommendations from a leading expert system (which not only can’t know when it is wrong but how much it can be wrong the few times it is wrong) and an average, experienced, professional in the domain (who can know how likely they are right, and if they are not likely to be right, how far off they are likely to be).

But even worse than the big data hype is the big platform hype … how mega platforms backed with cognitive abilities can do it all! They can do a lot, but they can’t do it all. And any delusions we might have that they can are only going to get us into trouble. Because as soon as we start trusting them blindly, we’re going to turn two blind eyes and that’s when the 2% failure rate is going to kick in, and materialize in the absolute worst way possible.

In Procurement, it’s going to miss the fact that a new organizational vendor is a very high risk and make a 2 year sole-source award for a small, but critical, custom made component in your (engine/control system) assembly when, in fact, it should be excluding the vendor which just had its credit score downgraded from a B+ to a D-. It’s not going to predict that in all probability, the vendor is not going to be able to secure enough loans to stay afloat (until it fulfills your orders and other customer orders and grows its business after losing a major contract that accounted for one third of its production) and will shut down and stop delivering product in 3 months when no one’s watching. Your production line will go down for 2 weeks while you find a backup supplier to quickly bring a production line up, make a minimal order, and air-freight it to you. If it’s a big automotive production line that goes down for 2 weeks, that’s easily 10M down the drain, and that 1M you saved is wiped out ten times over in a second.

But that’s not the worst thing that can happen from us turning two blind eyes. In this situation, the company temporarily loses some money, and as long as it has enough inventory to keep most of its customers happy, and it can keep its failure quiet, no one notices. Now, if its a medical diagnostics vendor and a (visually-based) diagnostic expert system (designed to help with the identification of all skin conditions) used by a remote doctor fails and mistakes melonoma for a relatively benign lesion, it’s a whole different story. When you consider that skin cancer is one of the five fastest spreading cancers, by the time the patient goes back in and insists something wrong, it could be too late — the spreading could be too far and the patient will be doomed (since melonoma, while only 1% of skin cancer, is not only one of the fastest spreading skins cancers but also has the highest fatality rate and causes the majority of skin cancer deaths each year).

Big Platforms give Big Confidence, but it’s false confidence. I’ll take a real human expert any day. Yes, she’ll make a mistake sometimes. But she also knows when she’s not sure and you should get a second opinion. That’s not always something a system can tell you. It’s above a threshold, below a threshold, or on the line (and no decision is made or classification is given). But that’s not always the right way to look at a situation.

* And, FYI, hiring college drop-outs whose college experience consists of cutting and pasting HTML and javascript code and fiddling with it until it works is not a good technology decision. There’s a big difference between being able to code a web-page and develop a highly scalable, reliable, and efficient enterprise computing system. BIG DIFFERENCE!