Category Archives: AI

GEN-AI is Failing 95% of the time. What does this mean for you?

We’ve known for a while that

  • Gartner’s first study found 85% of AI projects were failing (and that statistic is still being quoted everywhere, including this recent Medium Study)
  • Bain’s study last year found that 88% of all IT / technology projects fail to some extent (2024 study)

And we now know, thanks to MIT, that

  • 95% of all Gen-AI pilots fail. (Source: Fortune)

So what does this mean for you (and your ProcureTech journey)?

Well, beyond the obvious that you should stop dead in your tracks when a vendor starts pushing their “Gen-AI” enabled solution and dig deep into what that really means, at a foundation it means that:

You should never, ever, ever buy or use any solution that uses third party Gen-AI / LLMs, even if wrapped nicely, in their service or product because your chance of success will be 5% if you go with that provider.

You should only select vendors who only use in-house Gen-AI / LLM solutions that are built with the following rules in mind:

  1. custom trained on an expert culled corpus
  2. for a specific problem domain
  3. and applied in a specific context with guardrails and human checks on the output.

The best AI technologies has always been focussed on a specific problem, and this iteration is no different. Focus minimizes the LLM hallucinations (which cannot be trained out as they are a fundamental function of the technology) and guardrails prevent them from automatically being executed on / slipping through.

While they are far from perfect, with more discoveries being made daily on their many (many) drawbacks (where we summarized a dozen in this post on what not to do if you got a headache, but missed the recent revelation where it can not only lie on purpose but turn into something evil), the reality is that, as we have said before, LLMs, properly trained on vetted corpuses, do have two valid uses:

  • large corpus search and summarization
  • natural language translation

since, when appropriately trained, they can be almost as accurate as last generation semantic technology systems, but provide much more natural interfaces for the average user. (However, you won’t get a failure code from them when they are wrong, you will get a hallucination which will be so well phrased you’ll think it’s true when it’s an outright lie. Hence the need for guardrails and human review.)

So, if the vendor is

  • using their own in-house LLM
  • following the rules above
  • and targeting the LLM at natural language problems LLMs are actually good for

Then you should definitely try what the vendor is selling. (Try, not buy, and definitely don’t make a decision off of the carefully crafted demo!) Put it through its paces in a typical use-case for your company, not the use case selected by their demo master. If it does the task better on average than an average team member or does it about as good but many times faster, that is what you are looking for in a tool. Since there is no real AI, you can’t be replaced. But as your bosses keep increasing the weight of your workload to hit ridiculous revenue and profit targets, you need a tool that multiplies your productivity. One that can do the majority of the tactical data processing grunt work, leaving you free to do the strategic thinking and then add in the intelligence to a process or output that no tool can possess, instead of spending 90% of your time doing data entry, processing, and summarization that computers were built for.

In something like Procurement intake, that’s not trying to mimic in text chat the old school phone conversation that took you fifteen minutes to do the monthly office supplies re-order, that’s asking one question:

What do you want to do today?

processing the first one sentence answer:

Place the monthly office supplies re-order.

to determine that the user needs to be pushed into the e-Procurement system with the monthly office supply cart pre-loaded, so that all he has to do is enter the number of units of each item, and possibly add or remove an item from an easily searched catalog if one or two items need to be changed. Not 20 questions of “what do you need”, “what quantity”, “the same supplier”, “so you want 2 cases of paper from office depot”, “no, office max”, “oh, standard printer not glossy for marketing”, etc.

When Gen-AI mania first swept our space, and every vendor was told they needed a conversational interface for buying (or no customer would consider them in their RFP), and then built one, not a single one wasn’t painful to use. Most customers upon seeing it for the first time (after insisting on it), quickly said “can we turn it off” because they quickly realized that a well designed catalog with blanket/standard orders, quick search, and easy drill down to preferred suppliers was at least 10 times faster than trying to use a dumb chatbot — especially if they could pre-build templates / carts / blanket orders for regular purchases.

It’s the same for almost every other process vendors have been trying to apply this technology to, including conversational analytics. (Which, FYI, even Gartner expects to disappear from the conversation in two years.) There’s no such thing as conversational analytics, only reporting. And while that is really useful in the right context (such as allowing an executive to retrieve some basic information with a plain English question), try building a detailed spend cube, which is the cornerstone of spend analytics, with conversational analytics! (And I mean try because you will fail.)

While this doesn’t mean that LLM technology doesn’t have uses, it does mean that those uses have to be finely tuned. So far, among the hundreds of companies I’ve seen over the past few years, only a few have both implemented LLMs and gotten it right. Let’s hope that number increases in the near future. If, not always remember, while it would be great if a few more companies would get it right, You Don’t Need Gen-AI to Revolutionize Procurement and Supply Chain Management — Classic Analytics, Optimization, and Machine Learning that You Have Been Ignoring for Two Decades Will Do Just Fine. Not to mention the fact that good, adaptive, RPA will take care of most of your automation needs!

Why You Should NOT Engage Any Vendor Selling “AI Employees”!

It’s not just complete and utter bullcr@p, but it spreads a dangerous myth while demeaning and degrading all of us!

Complete and Utter Bullcr@p

As per AI Employees Aren’t Real! Don’t Believe The Lunacy:

  1. There is no Artificial Intelligence.
  2. An Employee is a Person!
  3. Fully Autonomous Software Agents Don’t Work.

Nor will they ever work with current technology as the existing algorithms, stacks, and technologies are not emergent, as has been proven, nor will they ever become magically emergent.

A Dangerous Myth

Psychopathic CEOs have been investing in technology for years with the dream of replacing employees who need fair wages, benefits, reasonable working hours, safe working conditions, and other costly annoyances with technology that can work 24/7/365 without any breaks, rights, or complaints. Given that each evolution of technology has enabled whole new categories of data processing and analytics jobs to be mostly automated, they have convinced themselves they will reach their technotopia in their lifetime where they can replace almost all their office workers with AI. For the past few years, they have heard the increasingly ridiculous claims of the Gen-AI vendors that “with just a few more trillion for dedicated data center construction and bigger model development, AI will achieve emergence and be able to do the work of a PhD level human” and have been waiting for that day.

Now you have vendors falsely claiming we have reached that day, and that, for less than the cost of an employee (or team), they can layoff entire departments and replace those employees with their custom Agentic AI that will do everything the employees did, flawlessly, 24/7/365, with the ability to scale up and take on more workload as needed.

But nothing is further from the truth. For example, this tech:

  • can only be encoded/trained to handle known situations with appropriate responses; when an exceptional circumstance arises that is not in the encoding/training data, it won’t know what to do;
  • is not flawless if any part of it is based on (Deep) Neural Networks or Large Language Models; the former will have a maximum accuracy rate and the latter will be completely unpredictable as you can ask it the same query five times in a row and get five completely different responses and there is no way this can ever be trained out (it’s another fundamental property of these systems, as per recent research); and
  • only works well on tasks that are computationally oriented, not on tasks that are more emotionally oriented.

They are making false promises that is not only giving companies an excuse for mass layoffs, but an incentive for mass layoffs that will not only harm you (as you will be unemployed), but harm them and their relationships (when the tech pays a fraudulent mulit-million dollar invoice, allows safety checks to be bypassed, and replaces a long-term proven supplier with a cheaper imitator whose only goal is to extort as much as it can from the market before suddenly declaring bankruptcy due to CXO embezzlement).

Demeaning and Degrading

Even if you could swallow all of the lies by saying “it’s just marketing“, you shouldn’t because it is demeaning and degrading to all of us to equate a piece of software with an intelligent human and claim one can fully replace the other.

There is absolutely no question that a machine can compute better than we can. They were designed to be the ultimate computational machines that could flawlessly perform trillions of calculations per second, and that’s what they do.

There is absolutely no question that the machine can do certain tactical data processing and analytical tasks way better than we can or that they should be employed to do so. Moreover, the tech that allows them to do these tasks has existed for at least a decade, if not two, and workforce displacement was inevitable. However, displacement does not mean elimination, it means shifting towards more strategic, relationship, or manual tasks that computation cannot capture.

Accounts Payable departments were doomed to shrink (as we had invoice processing solutions almost 10 years ago that could, with the right effort, increase straight through processing rates to 90% or even 95%), statistical analysis and data reconfiguration departments were doomed to go the way of the Dodo (because the vast majority can be automated and what can’t can be consumed by the departmental analysts that need to do the analysis), and the need for Procurement Buyers was doomed to be minimized as time progressed (because you can automate catalog orders, standard RFQs, inventory replenishment, etc.).

But claiming that tactical computation can take over strategic reasoning, which “AI” cannot do because it’s not strategic (although it can compute inputs to well-defined models); that cold computation can replace warm relationships; and that dumb probabilistic mush can replace human intelligence is demeaning. Furthermore, pretending that you can replace a valuable human employee with a costly piece of dumb software is degrading. (It is considerably more costly than you think. See Joël Collin-Demers post on The Dirty Little Secret Behind Gen-AI Functionality Pricing on why these vendors are switching to outcome pricing, and the reason is to hide how costly this technology is relative to the return.)

Succinctly put, you shouldn’t put up with it. You deserve respect, and that is something that vendors claiming “AI Employees” are taking away from you!

This why Sourcing Innovation had to update it’s Product Review Requirements for the first time since posting them back in 2007! While it has no problem with Agentic AI (as long as it’s just enhanced RPA which we know works), and can even deal with Agentic Workforce (since it is doing a form of work), it cannot accept AI Employee and is drawing the line here because someone has to!

This is why, after 18 years, SI had to include as part of product review requirements that the vendor accepts that SI has a no Bullshit policy, which includes no (Gen-)AI Bullshit, and SI will NOT cover you if you make ridiculous or false claims (that are not backed up by live demos and/or case studies with a customer that will go on record); this includes, but is not limited to, claims of AI Employees that we have already debunked. It’s not going to peddle your panacea poison!

Dear Procurement, Your AI ProcureTech Vendor Is Out To Eliminate You!

Here’s the dirty little secret they aren’t telling you. They aren’t building “AI Employees” (which aren’t real, by the way) because they want to give you a better, more complete, team which is able to work 24/7 and constantly process data, run analysis, respond to supplier inquiries, and have fresh insights in your inbox in the morning.

They are building “AI Employees” to replace you! Here’s why:

1. You Are The Hardest Sell!

A. You are the hardest negotiator! It’s what you do for a living.
B. You do your homework! It’s hard for them to bluff that no one else does this or we’re giving you a great deal when you’ve done your research and know 3 other vendors have similar capability, you know the quoted pricing from all their competitors, and you’ve talked to your association members and know what they are paying for the vendor’s solution.
C. You’re smart, and you know it. The LMFAO sales tactics CXO ego-stroking doesn’t work on you, and you question everything that sounds too good to be true. You’re one of the few holdouts preventing the ChatGPT-dystopia that would atrophy your cognitive abilities to the point you’d fall for their half-truths and beg for their system (which takes us one more step towards their dark city vision of the future).

2. IT Tries to Kill All Your Selections

You’re the biggest threat to IT’s total corporate dominance on system selection and management. In most organizations, every other department falls in line and eventually uses IT’s recommendations (allowing them to stay with preferred platforms or vendors that give IT the best software and hardware toys for free), but you question everything. You don’t swallow the one-vendor / one-ecosystem BS, the big volume discount BS, or the we-can’t-support-more-SaaS BS because you know Open APIs allow for ecosystem integration, that it’s not volume discount but ROI, and that, other then providing the API keys to the systems that the SaaS you selected has to integrate with, there’s no ongoing support requirements for IT. As a result, IT goes hard against your picks and tries to turn the C-Suite against you, complicating and delaying the deal process (and the longer it takes to sell, the greater the [opportunity] cost of the sale, and the less deals they can close in a year. Remember, to them, it’s not about the delivery, it’s about the close).

3. Your Budget is Baseline

In most organizations, Procurement is not sexy and is still seen as the biggest cost center (because too many executives believe profit is entirely dependent on revenue when anyone who can do basic math should realize that if P=R-E, then keeping expenses down can also be very profitable, and if the cost of goods sold is 90% of R, i.e. R – E = 0.1R, then it would take 9 times the sales to have the same impact on profit as reducing costs by 10%). As a result, your budget is baseline and there’s no wiggle room there, limiting their profit margins if they sell to you. On the flip side, they see other big enterprise tech companies making 80%+ margins for tech-cr@p they haven’t updated in years and the tech-bros raising millions, or billions, for AI with nothing more than ethereal claims of future capability, and they want a piece of the action.

But how do they get that action? Well, build something you can sell to the two biggest budget owners in the corporation: the CEO and the CXO, both of whom have been marketed to 24×7 for the past 3+ years by the A.S.S.H.O.L.E. and have now been completely brainwashed into believing that AI is going to totally transform their businesses while allowing them to lay off 80% of their cost-center resources (which, in their view, includes you!).

They’re only marketing to you to sneak in through the back-door, get your blessing, and then when you help them get in front of the CFO and CEO, off comes the grandma disguise to reveal the big bad wolf that lies beneath — a wolf intent on eliminating your job and your entire department so they can get 1/3 of the overall Procurement budget for their custom AI employees that will do all the costly functions you do, do them 24/7, and increase savings by increasing spend under management because they can strategically source, quick-quote, or auction everything you need to buy. (Even though this is an extremely bad idea. If you don’t know why, read our article on why real Procurement Leaders Listen To Roxette!)

In other words, if someone reaches out to you offering you an AI Employee, slam that door in their face as fast as you can while screaming at them to never show their face again! Because, if you don’t, you might just end up unemployed on the street corner screaming about how AI is ending the world, because we already know the inevitable that will happen once companies start relying on technology that hallucinates, (purposely) lies, fails at math, commits fraud, compromises your code, proliferates extremist views, blackmails you, maintains hit-lists, encourages suicide, lets you die to save itself, contemplates murder, and makes you dependent to the point of psychosis (so if got a headache, don’t take an aspirin or query an LLM). [Hint: it’s not good. There’s a reason I’m not yelling loud enough even though I’m already screaming at the top of my lungs.]

There’s No Such Thing as a Data Scientist!

Last month, Koray Köse wrote a great must-read post on why data scientists and orchestration officers are part of today’s silicon snake oil sales people and not qualified to solve your supply chain problems (my less-than-eloquent rephrasing of his words).

He was totally right in his rant, but I’m going to go one step further. There is no such thing as a “Data Scientist”. You can’t do “Science” on “Data” you don’t understand. You just can’t.

I don’t care how many mathematical models, statistical techniques, or “AI” toolsets you think you know (see my previous rants there), that doesn’t make you a “data” scientist — that makes you a mathematician, statistician, or new age script kiddie. (No better than the cut-and-paste script kiddies that hit the scene on mass before the dot-com crash, if you are old enough to remember it!)

I say this as someone who would best qualify if there was such a thing — PhD in CS with a thesis in multi-dimensional data structures and computational geometry, industry expertise in (Strategic Sourcing Decision) Optimization modelling (in high dimensions), spend analytics, etc. etc. etc. I was doing “big data” (more BS — we’ve always had more data than we could fit in memory on the machine resources we had available) before that was a term too.

Koray is also dead on with respect to PhDs. Even 26 years ago, you did a PhD to prove you could (or to stay in academia), not because it added anything of practicality beyond what you’d learn in a Masters.

However, I have to fact check him on the 50% to 70% supply chain tech project failure, since the latest Bain study puts the tech project failure rate in general at 88% and most of the partial to full failure is with the big players and Big X implementers (which is the majority of supply chain projects). The rate is higher (unless he’s talking full failure).

I am also going to remind him that this problem resonates through Procurement as well, so please don’t found just another ProcureTech company either. There are well over 700 now (see the mega map, and probably closer to 800 now) and we don’t need 100 “solutions” for the same problem that are almost the same. Get across-the-board experience, or at least spend years working with experts that have it, when developing your solutions if you are coming from a pure tech background.

Cutting Edge Tech Is NOT Defined by the C-Suite, …

… Financiers, or the Marketers pushing hype (from the A.S.S.H.O.L.E.) at you 24/7.

Nor is it defined by the algorithms it uses, the software stacks it runs on, or the hardware stacks it makes use of.

Cutting edge tech is ANY and ALL tech that

  • solves one or more significant problems that not being solved by your tech today and
  • does it by automating at least 90% of the tactical data processes that can be automated

It can be based on the latest AI algorithm or twenty year old RPA. It doesn’t matter if it shines a light using a LAMP stack, if it is an edgy MEAN stack, an Austin Powers inspired Grails stack, or even a .NET stack (though the doctor personally shuddered typing those last three caps out). The entire point of enterprise software is to solve your problems.

The point of software is NOT to provide

  • an excuse for a C-Suite to cut his tech-bro buddy a fat check,
  • a new Tulip Market for greedy financiers who think they can score big and get out before the crash, or
  • marketers a platform to pump out pompous poop on a daily basis.

As Mr. Koray Köse penned in a recent piece on LinkedIn on how you are in need of cutting edge technology, the last thing you want to do is take your direction from the VC-pumped C-Levels who do nothing but repeat the marketing garbage they are fed, sometimes changing the baseline of the garbage mid-sentence!

You have to remember:

  • All the VCs and most of the PEs want to create the next unicorn and get rich quick overnight. So most of these VCs and PE firms are pumping huge amounts into companies with little to no product (to support their grand vision that even SAP and Oracle haven’t managed to achieve after 5 decades and trillions of dollars) and directing the majority of that to be spent on buzz-creating sales and marketing (and not real product development). After all, you don’t actually have to create anything beyond buzz to get rich — market crashes throughout history have proven that since Tulip Mania. (What was created there of value? Nothing. But hype made a few men rich and many men poor.)
  • Even though today’s LLMs are dumber than a doorknob (and demonstrate more than any previous generation of the tech that AI should stand for Artificial Idiocy), with performance degrading every iteration (because there is no more data to steal, and the AI engines are now training each other on regurgitated digital garbage), marketers are still taking storytelling to a whole new level (and we mean storytelling because ALL the claims are fake) with a spin that even the Spin Doctors of old would be envious of. (Little Miss Can’t Be Wrong now, right? They want to Make You A Believer so you hand over all your Money when you should Exit … Stage Left and Run To The Hills!)
  • This copy is being pushed onto the C-Suites of all of the other investments in their portfolio with assurances that it’s all true, and being similarly echoed to all of the CXOs that attend the conferences they speak at, the golf outings they are invited to, and the exclusive social gatherings they arrange.

Not one of these groups knows what you need, and two of these groups have absolutely no interest in giving it to you — their interest is all about getting your money, building the hype, inflating the value, and, hopefully, cashing out big before the next hype cycle and/or the inevitable market crash that’s coming.

The technology you need is the technology that is:

  • built with real world problems in mind,
  • tested on real world problems in real companies and proven to deliver, and
  • scalable and extendable to your operations and needs.

This type of tech is built over years and doesn’t use unreliable probabilistic AI at the core. (It runs on traditional, configurable, RPA that is 100% reliable and auditable. Now, this tech might employe AI to help with the configuration by analyzing your systems and processes and self-assessed gap analysis by recommending configurations for you to approve, and that’s okay, because it’s not randomly making decisions, its recommending options and letting you confirm or deny. It might also use SLMs for specific problems where they work a high percentage of the time to jump-start searches, documents, and processes for you, and as long as you retain full control and can accept, modify, or reject, that’s okay too. But everything is built on a solid core, with 100% dependable automation for all key data intake, processing, and pushes that is done without any manual intervention, appropriately calibrated to your business rules, processes, and goals.)

It’s also built slow, rolled out to a small group of beta customers or development partners, and hardened in the field before being rolled out en masse.

And, most importantly, it’s built by a company that is boot-strapped or frugally running on a shoe-string budget from minority SEED investors to get that first version up-and-running successfully in its first 10 customers before that company goes for any VC funding to scale up. A company that has the time to get it right before being under constant pressure to make demanding, if not impossible, sales targets.

Moreover, to have any chance of getting this software, you need to know three things:

  1. how to identify what you need that will form the heart and soul of the RFX,
  2. how to write a good technology RFX and analyze the responses, and
  3. how to identify the right companies to invite to the RFX.

What You Need

For Supply Chain, as Mr. Koray Köse points out, if you need help identifying your true needs and cutting through the noise, he can help you out with that with the eyes of a hawk, the skill of a surgeon, and the wit of a Williams (a Robin Williams). For Procurement, Joël Collin-Demers can slice through your organizational landscape like a hot knife through butter and let you know exactly what you need in priority order.

The Technology RFP

Every consultancy and their mascot claims they can help you here, but you need to be very, very careful.

  • many of their consultants are not technology experts and tend to prioritize features over functions, as that’s all they know
  • many of these firms have partnerships with the (mega) suite players, and you don’t maintain sycophant, sorry, Gold/Platinum/Diamond, status unless you direct a LOT of business their way each year, so they tend to try to fit everyone into one of these buckets
  • many follow the old consulting model of “give the client exactly what he thinks he wants” and don’t take the time to figure out what you actually need and educate you, leading to an RFP that is no better than what you would have written yourself, as they spend half their time questioning you, and the other half writing down your responses

For true success, you need someone who is simultaneously:

  • an expert in the domain,
  • an expert in technology, and
  • not incentivized to help any vendor whatsoever and, preferably
  • an expert in project assurance (but not always necessary)

If you need help writing that ProcureTech / Direct Sourcing/Supply Chain RFP, feel free to reach out. This is my expertise. And for some tips, feel free to start with our recent series on How Do You Write A Good RFP?

Vendor Selection

Very few analysts and consultants know more than a handful of vendors. The big firms focus on the big vendors who cut big cheques, which are the 20-ish same vendors you see in their maps year after year after year. They don’t know about the other 700. Only a few of us independent analysts go far and wide and actually know what is out there and how it can help you.

For ProcureTech, SI should be your first stop. It’s the site that gave you the mega map to open your eyes as to just how wide the playing field is. Moreover, if you need something really niche where the doctor doesn’t have the expertise, he’ll find the right expert to refer you to.

For SupplyChain, Mr. Köse knows a lot of the players. But don’t overlook Bob Ferrari of The Ferrari Group. As one of the original supply chain analysts, he knows all the players and what their platforms can and can’t do inside and out.

And if you reach out to the right experts and get the right help, maybe you can get true cutting edge tech that actually helps you!