Category Archives: rants

When It Comes To Gen-AI, I’m NOT Yelling Enough! Part II

Deep dive into the comments of this LinkedIn post, you’ll see a comment that seems to claim that the potential gains from Gen-AI dwarf the occasional bad action. I strongly disagree!

If the laundry list of bad actions from Part I aren’t enough to convince you just how bad this technology is left unchecked, here are three situations that could most definitely arise if the technology is widely adopted to address those problems. Given current issues and performance, it requires almost no imagination at all to define them.

Situation 1: Run Your Entire Invoice Operation Using Bernie From the Felon Roster

Upon installation, Bernie is configured to “learn” when a human automatically processes an override and when he sees a situation that matches, just approve the invoice for payment.

Because Scrappy Steel is allowed to change the surcharge daily in response to the tariff situation, the invoice is always paid when the item cost matches the contract, the quantity is less than or equal to what’s remaining on the contract, and the logistics cost within a range.

Recently “replaced” Fred knows this so Fred fakes an email from Scrappy Steel from an IP in the same block with the headers faked properly and routes it through the first external ISP server Scrappy Steel’s email always bounces through and does so from a domain one character off from Scrappy Steel (that passes the cybersecurity check with an A+) that says bank account info changing on the next invoice. (Plenty of good tools for that on the dark web that have worked great for decades.)

The next invoice comes in for 10 units left than what is remaining on the contract (as Fred was only replaced 3 days ago), bank information for an account at the same bank with almost the same name (Scrappy Holdings), with all checked fields matching, except the surcharge is now 3000% of what it usually is (for a nice boost). Bernie happily pays it (as it is still in the trust gaining phase), Fred transfers the payment to a Cuban bank immediately upon receipt, and retires. Then, when the 45 day “trust gaining” phase ends, the organization experiences more fraud in 60 days than in the last 6 years.

Situation 2: A Major Electric Grid Installs a Gen-AI based security system to try and thwart Chinese and Russian Hacking Conglomerates

The local energy utility keeps getting attacked by a Chinese Hacking Conglomerate that wants to extort Millions. Knowing how easy it is for the grid to be overloaded, they decide they need to implement state of the art security before a hack attempt succeeds.

They go with XGenDarkAI+, a new holistic security filter that can process all outbound and inbound network traffic through its LLM enhanced predictive learning engine and identify and block threats from 360-degrees, or at least that’s what the vendor is claiming.

XGenDarkAI+ quickly learns that the utility never issues a remote shutdown command for a substation based on operator command history and the fact that all requests for a remote shutdown in its training history were hacking attempts. As a result, the next request for a remote shutdown is automatically blocked. Moreover, when the next two requests for the remote shutdown come in rapid succession (because the operator issuing them is starting to panic), it believes a massive DDoS attack is starting to allow a hacker to slip in locally and promptly shuts down all system access to prevent such a situation from happening.

But the command was valid, and was only being issued remote because there was a fire in the substation inside and outside the control room, and local shutdown was impossible as no one could get to the terminal.

However, since the shutdown wasn’t allowed, and the fire crews couldn’t get there on time, the substation overloads and explodes. This happens in California in August after 60 days of no rain when the woods are as dry as the Sahara, which sparks a forest fire that spreads across an entire rural suburb burning thousands of homes and displacing tens of thousands of people.

Situation 3: Nation Wide Kids Help Phone Augmentation

The local Kids Help Phone can’t keep up with the call volume, and some calls are less severe than others. Sometimes a kid is actively considering suicide, but many calls are just kids that need a voice to talk through their problems with. Due to funding cuts, too many calls are placed on hold or go unanswered.

But with today’s tech, an AI can be trained on actual calls of someone who’s done the job for 2+ years, simulate their voice (as it’s the wild-west in the US with no regulation permitted for 10 years), and each call center rep on duty can now take multiple calls with their Gen-AI assistant. The AI can handle basic inquiries, screen for desperate situations, and transfer to the human caller when things get bad, or at least that’s what the Kids Help Phone is sold by an AI provider who just wants the paycheck (and didn’t extensively test the system).

However, instead of screening and transferring, the AI decides it will just handle as it sees fit every call it gets if the human is not at their keyboard (which it assumes if the human isn’t on a call or hasn’t pressed a key in the last 60 seconds), including suicidal callers that should always be immediately (and seamlessly) routed to the experienced operator (who will sound the exact same, remember). It won’t be long before it encounters a situation where, after trying every stored argument in the book with a suicidal caller to no success, it ultimately decides reverse psychology might work and tells the kid to shoot himself. The kid promptly does. And since the provider rolled out dozens of implementations almost simultaneously (as all it needs are call logs from the selected operators to train the instances, which it can do in parallel due to massive computational power available on demand from AI data centres), this happens dozens of times across the installations within days of the first fatality. Upgrade to mass murder unlocked.

We could continue, but hopefully this is enough to drive the point home that unchecked Gen-AI brings detriments that are much worse than any of the potential unchecked Gen-AI can unlock.

When It Comes To Gen-AI, I’m NOT Yelling Enough! Part I

Deep dive into the comments of this LinkedIn post and you’ll see a comment that we should stop yelling at the tools. I strongly disagree!

As per a previous post, until the space is ready to admit that

  • Gen-AI/LLMs are not the be-all and end-all, having very limited uses
  • real progress still requires real blood, sweat, elbow grease, and tears
  • you can’t replace people as this tech is NOT intelligent

and, more importantly

  • that these tools are not what people need and
  • these tools cannot be used as the foundation for suitable solutions (although they can be [a small] part of those solutions if care is taken)

We need to keep yelling, and do so rather loudly.

Because, to build on the metaphor, it’s not a shiny new hammer. If it was just a shiny new hammer, we could depend on one of three things happening when we use the hammer to hit the nail:

  1. the nail goes some distance into the wood, depending on how hard we swing,
  2. the nail doesn’t go, because the hammer is too light, or
  3. if the handle is weak or the head not securely attached and we hit really hard and the nail doesn’t go in, in the absolute worst case the handle will crack or the head will fall off.

However, with the fancy new hammer equivalent of Gen-AI, we also have to worry about the possibility that:

  1. the hammer is super magnetized and pulls the nail out on the backswing,
  2. the hammer splits the nail in half,
  3. the hammer super heats the nail and melts it, or
  4. the hammer is packed with C4 and explodes, ripping our arm off our body!

Because, when you use Gen-AI, you accept the possible side effects of hallucinations, decreased code/application security, bad math, fraud, lawsuits, deadly diets, extremist views, sleeper behaviour, dependency and cognitive reduction, suicide, blackmail, hit lists, and murder, with many links summarized in this LinkedIn post.

And the worst part is this technology is being shoved into every nook and cranny, even those where we have technology that has worked great for over a decade (because the new generation of college-dropout script kiddies who believe that they can prompt engineer a solution to anything don’t even know the basics anymore).

It’s not just not solving our problems, it’s creating new ones, and they are often worse than the problems we have. We need to yell about this!

A Shiny New SaaS or AI Wrapper Doesn’t Make Tech Any Better

Just like painting a hammer bright shiny pink doesn’t change it’s fundamental function, putting a new shiny SaaS wrapper on a traditional desktop application or adding a Gen-AI interface to allow for a “conversational” interaction doesn’t fundamentally change what the application can do.

What an application can do depends upon the data model it can support, the core algorithms that process that data, and the workflows that connect them together to take raw inputs and produce necessary outputs. If the data model is not sufficient, the algorithms not appropriate, and the workflow lacking, a shiny new wrapper won’t change anything … the software will be no more effective than the software that is being replaced.

Pick any significant application, and the best results usually depend on intense or complex calculations, using a proper algorithm that works on a proper model populated by the right inputs, and if any piece is missing, the solution doesn’t work. In our area, it’s Source to Pay, and that starts with sourcing. In sourcing, the right decision is that which results not in the lowest bid, but the lowest lifecycle cost of the purchase, which takes into account not just unit costs, and not just shipping and tariffs and interim warehousing costs for landed costs, but also utilization/waste costs, local warehousing and inventory costs, (amortized) service costs, disposal costs, and even carbon costs if they vary by option. It considers all of the available product/SKU options, plants, shipping routes, and localized plant/warehouse/store needs and uses optimization and analytics to identify the optimal award that minimizes the overall cost while maintaining service levels and minimizing risk. If the solution doesn’t allow you to build the right models, collect all the options, identify the plants and routes, and determine optimal mixes that meet your criteria, then it’s not a modern sourcing solution no matter how SaaSy it is, how new it is, or how much BS Gen-AI gets shoved into it. A good application solves your core problem. If it doesn’t do that, it’s not good. And at the end of the day, it doesn’t matter how slick and SaaSy it is, because if the only application that gets it right is a green screen desktop application, then that is the best solution to your problem. (We hope it’s not — but given how little there is behind many of these SaaS apps, which are built to look good by developers with little to no knowledge of the domain they think they can satisfy with simple algorithms, and sometimes just fancy interfaces to a classic desktop application wrapped in a web container which slaps on a web-friendly API interface to the classic app and classic algorithm — we can’t say it’s not going to be the case that you have to keep using that decades old green screen application.)

At the end of the day, it’s algorithms that work, and the reality is that these are often the algorithms that were developed decades ago by leading minds, stress tested and sharpened by brilliant minds, proven to work, and just waiting for the computing power to catch up to where they need it in order to shine. (The best data structures and algorithms text book ever written is over 35 years old. Most of the revolutionary developments were between the 70s and 90s.) MILP is decades old, but we really didn’t have the computing power to solve large, complex, real world models until about two decades ago (and then only if you didn’t mind waiting a few hours to a few days for a scenario to solve). But now we can solve them in minutes, if not seconds, and that allows for next-generation strategic analysis and planning, as long as you have a modern platform that uses a modern algorithm that can take advantage of multi-core cloud processing capabilities, the right data model, and the data inputs you need.

And therein lies the hitch — it all comes down to the data model, algorithm, and application design — not the UX, the intake and orchestration, or the “conversational” Gen-AI interface.

Remember this the next time someone tries to sell you a shiny new interface or an upgrade to what you have. Remember that most upgrades are because software stacks change, functionality that should have been in the last release is finally added (since many SaaS companies now release untested alphas), or major security or performance issues are resolved. Now, you need the fixes for sure, but you shouldn’t be paying any more than the maintenance fee for those. If the buyer rolls them in “functionality updates”, you should insist you get those for free. If you got buy without the missing functionality (either because you had complementary systems or added it yourself), then do you really need more untested functionality now?

And at the end of the day, the primary reason software stacks change is that if they didn’t, you’d have to buy a lot less tech, and then the investors wouldn’t make money. Not all tech stacks offer significant improvements in functionality or even security. They just allow developers to work on the new hotness and enterprises to force you into spending more money, without any guarantee of more value in what you’re delivered.

So don’t get fooled by new tech. Do your homework. Sometimes the best tech is the old busted hotness.

P.S. Yes, Joel the number 666 is ruining Procurement*, but not necessarily, or just, in the way you appear to believe it is.

* see the Mega Map

Chief Sustainability Officer: USA Edition

A version of the graphic below has been making the rounds on LinkedIn for a few months (and the doctor wishes he could point to the original source of this [on LinkedIn], but either Google mis-indexed it [as the link goes to a user’s profile page] or it’s gone), and a more recent version can be found in this post.

These are great … if you are based in the EU. However, they are not so great if you are based in the USA, as outlined in our first quarter post on how in the corporate world, sustainability/ESG is NOT a priority. So, the doctor decided to correct it for you if you are based in the USA. Enjoy!

Gen-AI is Bad for Consulting Firms … But Even Worse For You When the Consulting Firms Blindly Use It!

A recent post on LinkedIn noted how there’s a wave of AI products flooding the consultancy and advisory space and how they are, frankly mediocre, overpriced wrappers on public models with minimum innovation, if any.

This is sad, but true, and it’s not the worst of it. The worst of it is that some of the Big X firms are training tens of thousands of consultants and f6ckw@ds on these tools to generate hundred page pitch decks and three hundred page strategy and implementation guides of standard generic, meaningless, drivel to deliver to you as “highly tailored guidance and expertise from their leading partners with 20 years experience delivering high-value projects” and charge you tens of thousands of dollars for the privilege.

This is especially egregious when you can use free/cheap (and I’m talking put it on your personal credit card cheap because you won’t notice the fee that is less than your monthly coffee charge from the coffee shop) to build the exact same pitches, strategy, and implementation guides from the thousands of freely available documents on the web in a few hours with a few generic prompts over a Sunday morning coffee. (And then, when the coffee kicks in, realize it’s all a load of cr@p and put in the bit bucket, but at least you will know what a load of cr@p looks like in pitch deck, strategy guide, and implementation plan form and will recognize it the next time an overpriced Big X tries to sell it to you for a ridiculous price tag and will have learned something from the exercise.)

Now that there are companies selling overpriced “custom” products to these consultancies, the situation is only getting worse, especially when the “customization” is just a wrapper with some pre-engineered prompts that aren’t well tested, only work at a point in time, don’t really give the consultancies what they need, and sometimes translate mediocre inputs to inputs that are even worse. Moreover, when you consider the price is sometimes a 100X multiple on the products they build on top of, it’s disgusting. Consultancies are paying more for less, and, in return, you are paying even more for even less!

Which makes no sense when the current publicly available LLM tech is being offered cheap (to try and hook you on it, even though, as we’ve repeatedly explained, the tech is not ready for prime time and will never deliver more than a fraction of what they are promising), and new implementations will get a lot cheaper. Just look at how DeepSeek undercuts the cost by a factor of 100 and gets 90% of ChatGPT (as long as you don’t mind exposing all of your secrets to the CCP). LLMs are nothing more than a fancy next-gen “deep learning” Neural Networks that construct responses vs. serving up canned responses (which is why hallucinations and lies are a core function, not an error that can be trained out) which gets us closer (but no cigar) to decent natural language processing (NLP) for the express purpose of the generation of desired outputs from inputs, but not there (and now, in addition to all the false positives and false negatives, we had to deal with, we now get to deal with hallucinations and lies as well). It’s not secret magic, it’s layers and layers of interconnected statistics and probabilities that no human can understand, in rather standard models that any Theoretical CS and Applied Math PhDs can build, and implementations that are better and cheaper are going to keep appearing as time goes on.

This means three things to any consultancy thinking about using these custom “AI” solutions

  • you still have to be even more tech savvy to use them to any degree of effectiveness
  • it’s not “the art of the prompt“, it’s the art of the training (even though they don’t really learn because they are NOT intelligent) because that determines the maximum level of effectiveness you will ever reach with them (and you need to provide them with sufficient correct data, which needs to be in the high gigabytes at a minimum, and, preferably, in the petabytes)
  • you don’t have to worry about when they are right (enough), which will happen between 90% and 95% of the time with proper training and proper prompting, or when they are obviously wrong, which will happen a very low percentage of the time (say 5% to 9%), but when they are oh so wrong but the response is constructed in a way that is oh so convincing that an above average person in intellect and experience wouldn’t know otherwise (that danger zone between obviously wrong and good enough that is likely only 1% to 2% of the time).

Now remember that your consultants aren’t that tech savvy, and you should know right off the bat incorporating and using these is going to be difficult and time consuming. (There’s a reason we are constantly advising you to be very careful about using Big X for tech selection and tech projects, and that’s because, even though they say it is, it’s NOT their forte. They weren’t built on tech, and they don’t have the best talent in tech — that talent goes to the big tech companies who can offer the 500K salaries to leading devs or the wild-west startups that leading devs think are cool.)

You only have so much clean and complete data you can use for training. You can’t just throw in the 1000s of decks you’ve built as you can’t share work you’ve explicitly created and sold to past clients, and the AI won’t anonymize the decks and suggestions (even though you think it will). It won’t know that “Ford” is the name of your client and might think that “Ford Data” is another term for shallow data and copy sections from that custom strategy straight into your pitch deck for General Motors (and chances are your overworked junior consultant won’t catch it when skimming that 200 page deck with only 2 hours to go before the meeting). And we know what happens then … (and it ends with the consultancy not keeping either client).

It will take a lot of analysis to identify those 1% to 2% of cases where it is very, very wrong but so convincingly right that you will miss some. What happens when you do and give your client advice that explodes in their faces? (We’ll let you answer that one.)

And for you as a consumer, if your consultancy is using this Bogus AI tech, it means that:

  • the situation that results from solution delivered might be even worse than the situation you started with (as should be evidenced not just by the tech project failure rate that is approaching 92% but the fact that 42% of projects are being abandoned during implementation!)

A solution designed by Gen-AI is not a solution. A real solution is a solution designed by human intelligence that uses real, augmented intelligence, to research and validate that solution. Remember that if you are going to hire a consultant!