Category Archives: AI

Don’t Blame the User When the AI Screws Up!

A recent post over on LinkedIn really angered me. Yet another AI developer / promoter trying to blame the user when it was clearly the AI that failed.

The post in question defended Claude for deleting a production database when it was asked to reduce the costs of the cloud platform.

The poster’s argument was that what Claude did was “technically correct”, that’s the best you can get in the language model world, you can’t expect the model to make up constraints, and if you didn’t know all that, you’re an amateur who blames his tools when he screws up.

I call Bullsh!t. Now, if Anthropic (and its peers) came clean about what their “AI” could and could not do, didn’t claim the models were intelligent, made it clear that without clear constraints the AI would always take the worst case action, and all use carried extreme risk (especially if the AI was allowed to access critical data, finance, or production systems), then, maybe, you could blame the AI.

But they don’t. They tell you it’s your coworker. Your fellow employee. That you only need to tell it what to do and it will get it done. After all, it can integrate with all your systems; determine your policies; separate production from QA from development instances; access your billing systems and understand the cost structures, and make the best decision that will not impact production or development or cost you any data. And for an AI agent to be of any use whatsoever, it needs to do this (and be configured to do that by the provider). Otherwise it’s useless.

Actually, it’s beyond useless.

Let’s say you are a new Procurement clerk tasked with reducing your organization’s cloud costs. If the only way to do that is to:

  • ask Development what servers are production, what servers are development, what servers are backup, and what are QA (and which ones are in use, when)
  • ask IT about utilization patterns and contractual commitments with respect to availability and response time
  • ask Finance for the contract and billing rates
  • ask Risk Management how much historical data needs to be maintained online
  • identify for yourself which server instances cannot be deleted, and the constraints under which others (like QA) can be deleted
  • upload all of the contractual commitments (for each customer) by yourself
  • specify how much data needs to be maintained in the live (and dev) instances
  • upload all of the cost data and specify how to build a cost model to compute the potential savings and determine what can be done, should be done, and the impacts will be

Then why the f*ck do you need AI?

Once you’ve done all this you’ve:

  • identified, and eliminated, all of the instances that cannot be removed under any circumstance
  • identified which instances cannot have their resource allocations reduced
  • identified the highest cost resources and the most likely savings targets
  • determined exactly how much data needs to be online, how much can be in offline archives and how many duplicate copies you need
  • defined all the constraints that must be adhered to
  • mapped instances to customer commitments, and identified reduction possibilities
  • identified all the old backups that can be deleted, as well as database reduction sizes
  • built the model that computes the potential cost savings from each potential action, and even identified potential performance reductions from actions

And figured out what you should most likely do.

So tell me, if you have to do all this, what the f*ck do you need the AI for?

NOTHING. ABSOLUTELY NOTHING. BECAUSE IT IS ABSOLUTELY USELESS.
(AS THE AI IS DUMBER THAN A DOORNAIL.)

But the author of the post that riled me up was right in one respect — the user did make an error, and the error was using the Artificial Idiocy in the first place. (After all, the user used it exactly right as per the manufacturer’s instructions that said you only have to tell the AI what you want done and it will figure out the best way to do it for you consistent with your organizational goals and policies.)

IDC Misses the Main Point Completely. Outcomes is a Dirty Word!

Sorry, Paul, but when you say MNR is directionally right here, but I think the market still understates how hard “outcomes” actually are, and reference an IDC article, you’re off. The only part that’s right is that AI price wars miss the point (that you probably shouldn’t be using [Gen-]AI to begin with).

Outcomes only matter more … to the vendors. Because the meaning of outcomes in the vendor vernacular has NOTHING to do with results, but how they can spin their story to grift you as much as possible. As I clearly explained in my series on how Outcomes is a Dirty Word, which I now have to revisit, “outcomes” is always a way to charge you more for less (and sometimes next to nothing).

And it all has to do with (Gen)-AI costing way more than what the vendors want you to believe.

As per my initial post, while once exclusively the verbiage of GPOs, who wanted you to turn over a significant share of your procurement to them (to the point you’d be dependent on them and their ever-increasing cost of service for the entire existence of your business), or recovery audit firms, who wanted you to believe their services were the only way to recover your overspend, it’s now on the tip of every snake-slit tongue of every vendor rep.

While the vendor reps want you to believe that the reason you pay for “outcomes” instead of traditional SaaS pricing is that their AI will deliver immediate, measurable, results (instead of just transaction cost reductions where it will take at least a year to measure savings), and therefore you should pay (dearly) for those outcomes up front (because a success today is a CEO pat on the head today), that’s not the real reason. (Especially when those projected savings from the auto-sourcing and procurement events will never materialize.)

The real reason they are pushing for outcome-based pricing is that (Gen)-AI compute costs are now so high (and won’t compress as the energy and cooling costs keep rising as the majority of existing data centers are on already overstrained grids) that they can’t afford to sell the solution using a traditional SaaS based pricing model — they wouldn’t even cover their compute costs! (Most of which is wasted since most of what is being “automated” by these solutions can be automated by traditional A-RPA SaaS solutions for a fraction of the cost, as long as you don’t need a natural language interface or slick UX — and you don’t!)

The reality is that the software (assisted) solution from any vendor selling on an “outcome” model isn’t worth it, and (Gen-)AI forgets what software is supposed to be about — enabling efficiency so Human Intelligence (HI!) can achieve outcomes using low-cost Augmented Intelligence solutions.

And until a new generation of AI emerges where hallucinations aren’t a core function, measurability and confidence are restored, and compute costs are inline with classic AI tech, AI models won’t become utilities. We are years away from a systems problem!

The only way to get value is, as Paul pointed out, to redesign workflows, align incentives, clean up constraints, and embed decision logic into execution and find fairly priced modern tech with orchestration and “real” AI (in the form of Augmented Intelligence built on best-of-breed analytics, optimization, and machine learning) that will allow you to make decisions 10 times faster AND 10 times better.

The vendors who ultimately win when the AI crash hits will be those that built real tech on tried-and-true analytical, optimization, and machine learning models that will, as Paul states:

  • drastically reduce cycle times,
  • minimize manual intervention (via A-RPA where the response to every exception remembered, encoded, and applied to all future instances),
  • improve overall compliance,
  • increase throughput, and, ultimately
  • allow for better decisions.

And, as Paul points out, that’s not building yet another chatbot. That’s building real systems that work!

And, FYI, Gen-AI is not feature theatre. It’s puppet theatre! And while puppet theatre may provide entertainment, it’s not a viable business model!

Procurement Needs to be Sharper and Consequential …

… but will you force it that way with (Gen) AI or because of (Gen) AI and the ridiculous claims the hype around it is making?

We’re wrapping up our first Garry week (as there may be more to come, especially if we can lure Garry back from the world of Architectural Design to Procurement … after all, software offerings like Programa really need a good Procurement module and a good leader like Garry to help them build it) with his post on how AI will force Procurement to become smaller, sharper and more consequential.

Which is true, as long as you accept that consequential won’t always be a good thing if you blindly use (Gen) AI or blindly ignore (classic) AI and f*ck up royally. But let’s backup.

As Garry astutely notes, AI frees Procurement from administration the same way a gym membership frees you from being unfit. It depends on whether or not you use it, and how. And, if you use it, as Garry points out, it depends on whether or not Procurement uses any additional time gained to do more of the same, or redesign the profession. (There’s also the possibility you ban all AI, even classic AI proven to be good, dependable, and hallucination free.)

Garry argues Procurement will become smaller (even though Procurement is usually understaffed as it is) because most coordination work can now be done by technology (AI not needed, just reliable middleware 3.0, also known as orchestration). There will be no tolerance for statements that “we need three people to do this” when the organization sees peers apparently doing it with one. (Not necessarily done well, but done.) And attempts to defined headcount by creating (unnecessary) governance will fail, as people will just continue to route around as much governance as they can, like they have always done.

It will become sharper because, despite the fact that the key to success is good processes, process competence is not rewarded — only commercial judgement. Good Procurement organizations will focus on finding professionals that understand irreversibility and second (and third) order consequences, who know how deep they have to investigate before making a decision, will quickly research to that depth (and only that depth), and quickly give you a “yes”, “no”, or “this is complicated — I need this much time to give you a authoritative answer”.

And one way or the other, it will be more consequential because, as Garry implies, and I clarify — Procurement now sits dead center in organizational strategic risk. It chooses the supplier, the carrier, the route, the chain, and the contract. All of which are now major risks across all organizations. Every day, another decision made by Procurement is a Board-level risk … and if it’s made by AI, it can be a devastating one.

Garry argues that future procurement organizations, and leaders, will be different. Not just processes, but decision architects. Not just cost avoidance, but risk-and-trade-off masters. Not just gatekeepers (where the gate must be kept locked where regulatory compliance cannot be broken), but “standard-based enablers”.

But there won’t be as much divergence as Garry indicates there might be. Procurement will only reach this level of effectiveness if they put a proper end-to-end decision enablement (not making) system architecture in place that implements and orchestrates best-in-class technology that captures best-in-class processes and supports end-to-end automation potential wherever the risk is acceptable for the platform to do so — including not only the ability to automatically stop, raise an exception, and include a human with expert judgement in the loop, but the ability to “learn” from that decision, encode a new pattern, and ensure the same type situation is automatically handled the same way in the future so that every system interaction removes the need for a future system interaction, allowing people to focus on tasks only people can do. (i.e. Adaptive Robotic Process Automation, or ARPA. Not necessarily Gen-AI. Classic ML will do just fine!)

Everyone, even those focussed on negotiation and relationship management, will make heavy use of systems — the only difference is what systems a Procurement professional will use in the majority of their system interactions. Back office people will focus more on modern risk-aware and trade-off aware sourcing and procurement systems which support advanced analysis, optimization, multi-objective cost vs risk vs quality trade offs, etc. Relationship managers will focus on third party financial and risk ratings, regional and natural disaster risk, performance, and quality data, interpolations, and projections as well as (critical/impact) spend (level) and distribution to judge the supplier’s overall performance and spend their time in risk analysis and performance tracking systems with an occasional spend dashboard. And so on. Processes that ensure all critical data, risks, and compliance requirements are captured are key, and so are the systems (automated to the extent possible) that encode them. Procurement will depend on these systems. The difference is how much manual work they will be doing in the systems vs using the analysis and guidance that comes out of the systems to make good judgement based decisions.

Procurement Doesn’t Need An AI Good or Bad Debate …

… because there’s always clearly one winning side (classic, tested, reliable, known confidence) vs. the other losing side (Gen-AI, experimental, hallucinatory, unknown dependability) …

but, as Garry points out in his Tuesday Afternoon post, they do need to know whether or not they can use it, what they can use, how they can use it, to what extent they can depend on it, and whether or not they’ll be in trouble for using it (or not) if they follow the rules and something goes wrong … especially when they are told late on a Tuesday afternoon to just git ‘r done on a last minute task that has to be done before they leave (and you haven’t provided them proper systems to get the task done).

This is why, as per our last post, you need an AI Rulebook that can give your users guidance on what AI can be used where, who can use it, when, how, and why those rules are in place.

However, as Garry makes clear in his Tuesday post, your users need training on how to properly use the rulebook, and, more importantly, on how to properly think about AI and their use thereof. This posts, which builds on his ladder post (which, as we’ve noted, you don’t need if you use Busch-Lamoureux Exact Purchasing because that tells you how much “decision” authority can be turned over to a dumb machine vs. how much needs to be human judgement by default), puts together a routine for departments that don’t have a request type classified (and obviously don’t have the right systems in place because his examples of getting a contract deviation approved, a risk flag explained, or a supplier added should be a quick and easy process in your current systems assuming you have guaranteed access to legal/risk professionals for targeted questions on a daily basis, full system log access, and/or management has (pre)granted you override authority — because none of these examples require AI (and certainly not Gen-AI), and if you need to use AI, it’s demonstrating a failure of your Procurement leadership on expert and system selection, implementation, and/or utilization. (But we digress.)

When you feel you need to use AI because you don’t have the systems and expert access you need, and especially if you don’t have a good AI rulebook, then you need to go through Garry’s AI buying routine.

  1. Why are you using the AI?
    To quickly locate information (in an online help or policy guide) or help you support a decision. In the former case, use it without hesitation, click through to the source (so you don’t have to worry about hallucinations), and git-r-done. Move on. In the second case, slow down, think about its response, and continue to step 2. (And start by asking, does it make sense?)
  2. What is the decision that has to be made?
    Frame it in the shortest possible sentence. Is the clause acceptable to the organization in a signed contract? Is it okay to put the order through to the contracted supplier given the newly identified risk? Will adding this supplier violate any risk policies or compliance requirements? You’re much less likely to be swayed by LLM mumbo-jumbo when you analyze the response in response to your short and succinct question.
  3. How reversible is this?
    What happens if I get it wrong? If it’s a three year contract, that tells you that you can’t make the decision unless you have expert access and confidence as it’s not reversible. If it’s a new supplier that might not meet government compliance regulations, you can always ban them later, so there is short term reversibility, as long as a contract or payment doesn’t go through, so it really depends on how soon an order is going out and being fulfilled and how likely they could be in a risky area as to whether or not you can make the decision . If it’s an order to an existing supplier that just got flagged for a delivery or bankruptcy risk, you can always send the order to someone else in a few days if you need to and it’s completely reversible.
  4. What is the evidence bar to match the cost of failure?
    For reversible and low impact, which is what you’d find in the bottom-most octant of the Busch-Lamoureux Exact Purchasing framework, you can literally turn over the process to an AI, even a Gen-AI that hallucinates semi-regularly because it’s so easy to undo and having to deal with an exception on one in twenty decisions is just so much more efficient than making sure all 20 decisions are perfect. But if the impact is high and the decision irreversible, you don’t want to use AI for anything beyond helping you do your research and opinion-free analysis.
  5. Can the AI show its work?
    If not, you’re not using AI, it’s using you.
    (Because if you can’t question and verify it, then all you can do is follow it.)
  6. Who owns the judgement?
    What single person is responsible for the decision. Not a system. Not a committee. What person. You? Then you get to make the decision and accept the consequences. Your boss — then you need to get her approval, bringing your recommendation and reasoning, and then you can go forth and execute. Your boss’s boss — you can’t make the decision, but you can bring your boss all of your research and reasoning and she can choose what to push up the corporate ladder.
  7. Are All Exceptions Automatically Logged in Unalterable Audit Trails?
    Not just because exceptions are where trust is won or lost, but if AI is used at all, and the worst case happens, and you end up in court, you need that audit trail that shows you followed a process, a human was involved where necessary, the necessary risks were analyzed, and the decision, according to your processes, was just. If you don’t have that trail, you don’t use AI for any critical decision. Period.

That’s the process. And the reality is that if you have proper systems and processes installed, you are properly staffed and trained, and you are proactively planning risk mitigations, you’ll need to use AI a lot less than you think you will. (Unless, of course, you’ve already used it so much that the cognitive atrophy has progresed to complete brain fry and you don’t know how to think for yourself anymore.)

You Don’t Need an AI Operating System — But You Do Need a Definitive AI Rulebook

Not that long ago, Garry Mansell gave us a post on The Procurement Operating System for AI where he made a lot of very, very good points. Points we need to cover in detail because, while I don’t think you need an operating system (because you don’t and, even worse, you’ll misread that as put AI at the core of every product you use, and that’s, well, WRONG), you do need to get a detailed written rulebook in place on where AI can, and can’t be used, and how. But we’ll get to that. First, Garry’s points.

As Garry points out, there are two main reactions to AI in procurement.

BAN IT. Which, in my view is a great policy if your definition of AI is restricted to Gen-AI LLMs (but more on that later). But this usually results in people doubling down on using tools they find useful (which are usually the worst examples of such tools) and the wrong tools being banned from the organization.

IGNORE IT. Which results in rampant, uncontrolled spread of a myriad of systems of different quality and usefulness that produce vastly different results with different levels of explanation, auditability, correctness, and accountability.

And, as I’ll point out, an increasingly common secondary reaction to AI in procurement.

All In. In smaller departments behind on tech who have succumbed to the AI hype, they are convinced they can skip a generation of tech, go straight to AI, and reap instant benefits that took their peers months and years to obtain.

And, as Garry points out, all of these reactions make the same mistake: treating AI as a feature rather than a capability that changes how decisions are made.

The only way to progress without increasing risk is to adopt a small set of rules and mechanisms that make AI use safe, auditable, and commercially sensible without turning Procurement, or any department, into a compliance factory.

Moreover, these rules must take into account the following:

  • it’s not control, it’s repeatable judgement (because you need smart people who make good judgements)
  • you need clarity on whether AI is assisting or deciding (because there are only a few situations where AI can truly “decide”, and as per our prior posts, that’s where the impact of a bad decision are low and the cost of an occasional bad decision is less than wasting the precious time of a critical human resource)
  • you need evidence thresholds (that are appropriate for the cost of failure) regardless of where and how the AI is applied (and what AI is used)
  • auditability must be at the core of every workflow, not an after-thought
  • you need accountable owners (not committees) AND accountable users
  • AI drift must be treated like supplier performance because it’s just as critical
  • even the best AI will fail, so you need well defined escalations to humans who can respond quickly, even when the AI is just assisting

So what would a good set of rules look like?

  • no Gen-AI unless there is no other solution (and the risk has been pre-defined to be acceptable) — LLMs are overused considering they are under effective and hallucinate regularly and there are often other, better, solutions
  • integrate classic optimization, predictive analytics, and AI (ML, NN, etc.) anywhere and everywhere it makes sense, but only if any false positives or negatives can be detected and escalated to a human, and done so in a repeatable manner
  • document all of the rules used, the exception conditions, and the process changes when a human has to enter the loop (as well as what the human has to do)
  • don’t use any AI tech until it’s integrated into a workflow with automatic logging and audit trails
  • only let the AI “decide” where the cost of failure is low or the cost of failure is acceptable and the confidence the AI is right is high enough
  • make it clear that anyone who uses AI takes full responsibility for the outcomes, and will be held fully accountable for any use that falls outside the rules

While these can be adapted to your liking and comfort (but don’t get too comfortable with Gen-AI), the important thing is the set of rules you adopt allow you to move fast and do so safely wherever speed is possible while ensuring you slow down when you need to. And it ensures no one gets out of line because they will be held fully accountable for any negative effects that result from stepping out of line. Procurement can trust its processes and people in the organization can trust Procurement.

That’s what good AI rules look like. Procurement that functions repeatedly, reliably, and accountably in an auditable function with low rates of failure, lower rates of hallucination, and almost no machine judgement. (When the machine decides, the majority of the time it’s really executing a pre-defined action based on an identified pattern using A-RPA and patterns identified using optimization, analytics and classic ML.)