the doctor recently went on a big rant about the analyst firms and the utter lack of usefulness in the maps they release, the focus they put on what they don’t understand, and the award categories they invent because, even though they have/had some great talent (and should be doing incredible work), what they’ve publicly released has been mostly valueless to the market they’ve been trying to serve (when it wouldn’t be too hard to provide a lot of value based on all the research and work they do). In the doctor‘s view, this is very sad because if they could demonstrate the value they provide, they would be more relevant across the market (and likely get a lot more business from smaller and/or more innovative providers who think that, because of the budgets the big players like Oracle, SAP, and Coupa have, the analysts are always going to recommend those companies anyway).

However, now he’s gone from sad to mad about something he has just heard from a couple of vendors regarding one of the biggest firms, because, if true, it means not only do they not have a clue about what is and is not valuable in tech, but they are unnecessarily creating confusing and obfuscating technology that still may be best in class.

So what have they done now? Well, apparently they are now basing 30% of the score on whether or not the vendor has “AI” in their platform, something which they’ve repeatedly proven they have ZERO ability to score whatsoever! So, either a vendor makes false, grandiose claims (and tries to use Applied Indirection to fool the Analyst Idiot that they have more than Artificial Idiocy in their Application Implementation), or they get scored low even if they have the best technology built on best practices, proven algorithms, and consistent results that give their customers a 5X to 10X ROI.

True AI adds value, but, in the doctor‘s experience,

• up to 80% of AI claims are Applied Indirection (at best) or Artificial Idiocy (at worst); in fact, some of the “AI” in spend analysis is still the “AI” they used in the early 2000s, and the doctor would rather not spell out that sad, but still true for some vendors, racial slur
• up to 80% of the rest, or up to 16% of tech that claims AI, is level one Assistive Intelligence; and this is typically just classic RPA (Robotic Process Automation) using human-defined parameter-based rules, and the “AI” is the automatic parameter adjustment based on user overrides … not very intelligent, eh?
• up to 80% of the rest, or up to 4% of the tech that claims AI, is level 2 Augmented Intelligence, which is the first level of AI where the tech can learn from human feedback and provide better insights and recommendations over time on one or more specific tasks, and the first level of AI that you should even consider as AI
• up to 80% of the rest, up to 1% of the tech that claims AI, and the highest level modern technology has generally achieved, is level 3, Apperceptive Intelligence, or Cognitive Intelligence, where the systems can not only learn from specific human feedback to recommendations but from general knowledge and intelligence available to it from integrated data sources to mimic the performance of the best human experts over time, even evolving processes, behaviours, and actions within well-defined bounds
• and then the rest, 0.1% or less, is nearing level 4, Autonomous Intelligence, where the system can learn, evolve, adapt, and maintain itself over time without human intervention … and hopefully execute meaningful, appropriate decisions grounded in best process and fact that considers all of the relevant information available (and not go off of the rails and advise you to commit suicide because you feel bad, Hail Hitler, or sacrifice a trolley full of people and a cross-walk full of pedestrians because there might be a cat in the road — all things AI has already done)

And even where a platform has semblances of real AI, chances are that the AI (the vendor is now forced to include or arbitrarily be relegated to the dustbin because, apparently, it’s not solutions but buzz-acronymns that matter now) is producing worst results than the best traditional algorithm or methodology on expert curated data sets and dimensions. For example, the vast majority of the market believes AI improves forecasting. It doesn’t. The best AI is still inferior to the best techniques developed in the 70s when applied to the right data dimensions. All the “AI”, which is just fancy, souped-up versions of classical machine learning (using algorithms developed in the 80s and 90s for which we didn’t have enough computing power until recently), does is run all of the data through a model that integrates classification with prediction to filter out the most relevant dimensions and the best curve fitting technique as all these algorithms, at the core, are based on 50+ year old statistics! This means that, at the end of the day, their best case performance is something a human genius figured out 50+ years ago.

But to achieve that best case, the developers have to implement the right AI algorithms, tune them properly, allow them to run long enough to correctly fit (but not over-fit) the training data sets, and monitor those algorithms over time … and to do that they need to be an expert in those algorithms, which they probably aren’t. So, in order to “check a box”, and sell you a product, they are ultimately integrating algorithms that will give you an inferior result (while requiring considerably more computing power that runs up your cloud utilization bill), versus sticking to tried-and-true algorithms and processes that their experts tweaked over years and that their experts can explain and verify at any time.

And this is an almost reasonable example of what a technology vendor might do (as the best predictive algorithms are not untested “AI” but based on classical, tried-and-true, statistical or optimization functions). Most of what the doctor has seen is MUCH worse than this. And the fact that some big analyst firms are now forcing vendors with good tech to integrate underdeveloped, unproven, and often untested AI just to get a rating, make a map, or be recommended is downright stupid.

SHAME ON ANY ANALYST FIRM THAT DOES THIS! Buzzwords are not products, and unproven tech is not value. Analysts should be recommending the best solutions, regarding of the tech they are based on. the doctor is simply appalled!