Category Archives: rants

In the Age of AI, the Analyst is King!

Never forget this!

(Click here to learn why DPGLabs.ai is best tool a VENDOR has to judge their perceived place in the ProcureTech market if that’s what you’re looking for.)

This rant is inspired by Dr. Elouise Epstein who, in this LinkedIn post, said the most wrong and dangerous thing she ever said. (And this SI article is an archive of my immediate rebuttal, found in this LinkedIn post.)

Saying that in “in the Age of AI, analysts are superfluous” is akin to saying that, “in the Age of the Internet, the Journalist is unnecessary” … and we all know how false that was, and still is. (Because the claim that the internet allowed everyone to be a journalist was not only far from the truth, but also ignored the fact that many people misuse the internet to spread rumours, conspiracy theories, and false facts that, when combined with FUD (fear, uncertainty, doubt), caused others to spread and amplify those same false facts. And if Gen-AI has proven one thing, it’s that it can spread and amplify falsehoods at the speed of light.)

(Now, this is assuming Dr. Epstein believes that an analyst does more than sticking random logos on a worthless map. But if that’s her definition of analyst, then maybe she’s right.)

There are multiple problems with her statement, and more problems than can fit in the 3K post character limit of LinkedIn, so we focussed on the biggies, as that should be more than enough to point out how Gen-AI has made the Analyst more critical, not less.

1) We are not in an age of true AI. We are in an age of hallucinatory Gen-AI.

This means that, even if all the data available is mostly correct, the AI can still get it wrong.

2) DPG Labs is NOT using evidence, they are using content from various URLs and weighting it (and, if the weighting is high enough, calling it evidence).

This means that if a vendor floods third party sites with unmarked advertorials filled with fake or misleading content, the vendor profile that Gen-AI creates is sure to be wrong.

3) Most vendors and customers blast success stories and hide failures.
(No one wants to admit they blew millions of dollars.)

These only come out in private conversations and the only “evidence” comes a few years when the customer switches to a different platform much sooner than the norm.

etc.

This means that no auto-generated profile can be considered reliable without human review. (And the odds of correctness range from 95%+ for the major vendors where there is lots of correct content to work on to 50%- for new vendors with little content or a lot of misleading/inaccurate content.)

But even if the profile is mostly right, tech cannot do the following:

4) Judge organizational fit.

That goes beyond a score. (That’s why Xavier Olivera and I designed the SpendMatters, now Hackett, Solution Map as a TechMap. We assessed what most organizations couldn’t, gave them average unbiased customer scores against that assessment, and let the organizations focus on what they could assess — their needs, the vendor’s culture, service, relationship approach, etc. — and not what a static map can’t do.)

5) Give you a true timeline and success rate.

Vendors over promise and under deliver, as they all assume perfect scenarios that never materialize. Dumb Gen-AI can just assess statements, not fact, and can’t understand what factors are involved in the 3 month best-case implementation, the typical 6 month implementation, and worst-case 12 month implementation for organizations of similar size, complexity, and geography in the vendor portfolio (where even the best case 3 month is still twice as long as what they promise).

6) Help you decide if you can trust the vendor.

This is probably the most important point. Do they have what they claim, can they deliver it, and are they committed to delivering it to you on time, schedule, and budget. And it’s not just their word, or a third party survey on a select customer sub-set, it’s based on realistic assessments of performance and the individuals assigned to your project.

etc.

That being said, DPGLabs.ai is the best tool a VENDOR has to judge their perceived place in the market.

It’s also the best foundation out there on which to build a true next-generation analyst offering. If this existed a decade ago, the efficiency gains it would have enabled would have allowed us to cover 80%+ of the vendors in depth at Spend Matters (vs 20%-) on Solution Map as it would have slashed our research and writing time* and allowed us to focus on verification, gaps, and uniqueness.

I described why the offering is the best tool VENDORS have, as well as how it could be the best tool to build a new analyst offering in the comments (of the original post) in detail in this LinkedIn post, where I have included the content below for easy access.

But even though the offering has a lot of value, it’s still nowhere close to a replacement for a true analyst.

The true value of DPG Labs.

After 20 years as an analyst, I’ve learned a lot.

This includes the fact that a lot of vendors don’t know how to position themselves, and know even less about how they are positioned in the market.

To this day, I still see new vendors regularly misusing purchasing, procurement, and sourcing when describing their offerings and wondering why they aren’t showing up in the right lists, maps, or articles.

I see vendors over-focused on the hype-du-jour (AI, Agentic, Intake to / Orchestration, Analytics, etc.) who completely ignore descriptions of rock solid capabilities or unique offerings that would differentiate them from the 40 to 100 other offerings that sound almost the same. (See the SI Mega Map if you don’t believe how many competitors you really have! The 7EB V1 edition has 888 vendor logos.)

I see vendors who have no idea how the market, and more importantly, how the search engines / AI tools used by the market see and position them.

What David Bush and his team have built at DPG labs is the perfect answer to that. By correlating and cross-referencing to a common vocabulary all of the external data and coverage on you against a standard functional map definition (that captures the questions buyers are likely to ask), you can see which categories / verticals / etc. the market is putting you in and which vendors you are being put up against. If those aren’t the vendors you expect to / want to be put up against, this means one or more of the following is true:

  • you’re not describing your offering appropriately
  • the coverage you’re getting is not focussing on the key / unique / valuable aspects of your offering (and you have to change your analyst relations / marketing / partnership approach)
  • you’re (trying to) compete in the wrong market

and you have rock solid proof you can take to the C-Suite that you need to market / sell / partner different if you want to succeed.

Up until now, analysts could only tell you this … and you/the C-Suite could ignore us (by claiming it was our opinion — which it was, but our expertise meant we were right the vast majority of the time). But this tool is essentially doing what pros using AI (enhanced) tools are doing to research you, and showing you where you stand.

It’s the one offering that I would have loved to create when I was at Spend Matters, but never could (because, even 3 years ago, the AI models just weren’t there to support it). And it’s the one tool you need the most. (Plus, market clarity will ensure you score better on the right maps since the analysts will have a better understanding of you before the first interaction, which will help them ask the right questions, focus the demos, and give you the right coverage.)

And even though you can use it to create a shortlist as a potential customer, you can’t use it for final selection. I explained why in this comment, where I also reiterated how great it would be as the foundation for a next-gen analyst firm. The core rationale is the following.

We have to remember:

  1. publication is NOT evidence, it could be opinion
  2. probabilistic assessment of language is not human assessment of language
  3. news coverage is often still marketing, especially when it’s an advertorial or there is paid coverage
  4. disgruntled customers can spread lies as well as truth
  5. even “verified” sources can be hacked or altered or replaced on the internet
  6. … and when (correct) coverage is insufficient, errors (and hallucinations) multiply in Gen-AI (frontier models)

Your (Gen-) AI tools don’t know any of this! Unless every profile you build is reviewed by an expert human AFTER an expert human review of the solution (i.e. a real analyst with real knowledge and experience), which DPGLabs is not doing (as they don’t have a real analyst team), you can’t be sure it’s accurate and not free of hallucinations.

P.S. This means the only profiles I’d trust on DPGLabs are the human reviewed profiles that Dr. Elouise Epstein reviewed, which would be the profiles of the vendors she knows well, and that’s a small minority!

* Not that Gen-AI produces good content, but, as per above, when enough material exists, it can create great summaries that allow an analyst to get a reasonable expectation of a vendor offering before the demo, produce summaries for the vendor to self-correct, and produce the auto-gen fact sheets, allow the analyst to focus their efforts and writings on the true analysis, opinions, and insights that you really need.

If You Can’t Do The Job Without Tech …

Then you can’t do the job with tech.

I’ve said it before and I’ll say it again. If you can’t do the job without tech, then you can’t do the job with tech.

All tech does is automate tasks and workflow processes and allow you to speed them up by a factor of 10 to 1,000,000 plus (depending on the tasks and workflow processes).

It doesn’t make tasks or processes better or worse (unless you Gen-AI, which typically makes it worse) — just faster (unless, of course, you change the process when you implement the tech and make the task or process better or worse).

Now, you’re probably asking why it’s not enough to design the process and simply install the tech. That’s because there’s a difference between designing a process and executing a process. Anyone can design a process at a high level given a highly set of requirements. Not anyone can execute. Among those, even less can execute it effectively.

Only those who can execute it effectively are fit to do the job, regardless of what tech the organization has or does not have. That’s because, if you don’t know how to do the job without tech (and certainly without AI), you don’t understand what the job really is. Business was conducted for thousands of years without modern tech, and computers only became generally available in big business in the 80s and mid-size businesses in the 90s and small businesses in the 2000s. That means everything you do today with modern tech was once done without modern tech (and, 150 years ago, without any tech that we would consider modern at all).

That understanding means you understand the core of the function. You know what has to be done, how, where the time suck is, what can be automated, how, and why. You also know how to verify correct automations has been implemented, and where the results require human interpretation, review, and decisions — and, if necessary, make those interpretations, review, and decisions.

Without that understanding, you can’t properly make use of tech and you definitely can’t make use of AI. Sure some tasks might be 10 to 1,000,000 times slower (and sourcing optimization will be out of the question, just a few bid comparisons), and you shouldn’t do them by hand unless necessary, but the point is you need to be able to — otherwise, you can never judge what the system does. Iin the age of hyper-fast LLM hallucinations, only good human decisions will allow your organization to succeed.

Why Your ProcureTech Initiative Will Fail … Part 2

We’ve written many series on best practice tech identification, tech selection, and tech implementation in the hopes of inverting the odds from an 80%+ chance of failure to an 80%+ chance of success, but given that failures are literally still happening on a daily basis, it seems that most people can’t be bothered to read best practice advice so today we’re going to flip the script and tell you all the reasons you’re going to fail and then you hope you go back and read the best practice advice we’ve freely given you (in series such as Successful Vendor Selection – The Series).

5. You Don’t Understand How To Select Proper Systems

As per our previous instalment, the answer is never to select the systems recommended by your favourite Big X consultancy whose recommendations are always in the mutual back-scratching club. With all the finders fees and recommendation fees built into the relationships, you’re paying multiples of what you should be for the tech even if the tech is appropriate. Typically the tech in the big vendors isn’t the best or most modern tech, and often the tech you fall back on only if you’re such a large enterprise that smaller vendors can’t serve you.

It’s not vendor size, Big X recommendation, marketing, or hype. And it’s never (Gen-) AI unless there’s no other solution. AI accelerates what you give it. And when you give it bad process, bad data, and bad instructions, you get an even worse result than what you have now. Until you’re best in class with last generation tech, you’re not even ready to consider AI.

The answer, as we’ve said before, is to find an independent consultant or consultant from a niche consultancy with no vendor relationships and no implementation team. The consultant/consultancies only line of business must be advisory, solution advisory, and project assurance — not implementation and integration and definitely not vendor partnerships. While the consultancies with the partnerships will tout the benefits and how they get (their customers) top tier support, first response, and the newest releases; that’s hype, not results. Results come from the systems that are right for you, not the systems right for the Big X consultancy.

Unless, of course, you are skilled enough to implement the best practice selection process yourself.

6. You Don’t Understand How Long It Will Take To Implement New Systems

A technology system implementation, replacement, or upgrade is an organizational mega-project in a mid-sized or larger organization. Even though SaaS vendors can partition a new instance in minutes, that’s not implementation, integration into your systems, incorporation into your daily processes, or institutionalization into your team’s routine. That all takes time. Lots of time — and a lot more time than your vendors will tell you. They know that you’ll be drawn to the proposal with the shortest implementation time so they will lie about how long it will really take and base their proposal on the theoretical minimum if everyone worked 12 hours a day, 7 days a week, and nothing ever, ever went wrong.

But we know the truth. Everything that can go wrong, will, to some degree. No one can dedicate as much time as the project plan says. All of the integrations will take longer than estimated. The data will be 10 times worse and 10 times as incomplete as was assumed, so there will be delays to clean the data and integrate external sources to buff it up. So things will drag out well beyond the proposal. Typically multiples.

That’s because no one will admit it’s a mega-project, and that’s why the technology project failure rate keeps increasing year-over-year despite being at an all time high of 88%+ in general, and 94% if Gen-AI. The reality is that mega-project success rates are 0.5%, as chronicled by Bent Flyvbjerg & Dan Gardner in How Big Things Get Done, where a study of over 16,000 projects across 136 countries going back to 1910 revealed the success rate. This is where technology project success rates are headed until everyone wakes up to the reality that they are dealing with mega projects and all AI does is speed up the failure.

7. You Don’t Understand How To Do Change Management

It’s not just implement the software, integrate the data feeds, flip the switch, and go. That’s a surefire guarantee for system avoidance and bypass.

Change management involves understanding how the processes are changing, what training the users will need, how best to go about it, how fast the switchover can actually happen, planning for, and managing all the non-technology details and implementing proper project assurance to make sure it actually happens. That last part is key — project assurance that starts before the first step of system implementation and continues until the adoption and usage hits the required targets — which will typically be months, if not years, after initial system implementation depending on the system. A few months for a dedicated solution or small suite, to a few years for an organization wide ERP or SCP for a large multi-national organization in dozens of countries.

8. You Don’t Know How To Recognize and Correct Issues To Avoid System Bypass

Just like no implementation will go according to plan (which is why over-optimistic schedules will never happen), no design will be as perfect as believed. Something critical will always be missed and multiple significant issues will crop up over time. The ability to recognize these issues quickly and do something about them before your employees start bypassing the system — because once they start, they won’t stop — the harder it will be to get them back on the system.

Let’s say you buy a new e-Procurement system designed to curb P-Card tail spend and allow the vast majority of spend to follow procedure, be tracked against budgets, and properly managed over time. Sounds great in theory, but lets say that any purchase not in the catalog requires an RFQ with at least 3 vendors, or a 3-way price comparison between 3 online vendors with a manager’s sign-off, which adds time and hassle for purchases that used to be one and done because the amounts were under thresholds, all of the options were always within a few % of each other, and saving $2 on a $100 purchase is NOT worth 15 minutes of an employee’s time who’s fully burdened cost is $100 an hour.

If you don’t pick up on this quickly, and let them one-and-done purchases under the old P-card amount through easy-peasy punch-out, they’ll go back to P-carding everything and if you take the P-card away, they’ll go back to the personal credit card and monthly expense report. You need to quickly figure out who is bypassing, why, and how you make the system easier to use than bypass.

The Procurement Knowledge Devolution

The internet was supposed to kick off the procurement knowledge revolution, allowing Procurement pros to quickly:

  • find out about best practices
  • get commodity and market pricing for products and services
  • build reasonable (should) cost models
  • find out about new suppliers, carriers, and consulting partners
  • research new products and services
  • hold truly global sourcing and procurement events in real time
  • effectively communicate with, manage, and develop suppliers
  • etc.

For twenty years, that’s what it was. Procurement departments who learned how to use the internet properly for research, deployed the right SaaS to support their processes, and identified the right data for their processes were successful in their endeavours.

But then came Gen-AI LLMs, chatbots were replaced with chat, j’ai pété‘s and clod‘s, and knowledge was replaced with whatever content the LLM generated. Maybe it was correct, maybe it was mostly correct, and maybe it was a 100% fabrication — a hallucination if you please. The problem with LLMs is that they are NOT intelligent. They are essentially super sophisticated multi-level cross-connected deep neural nets that go beyond classification to generation of responses built up from sub-responses built up from deep training on incredibly large data sets.

Therein lies all of the problems. It generates built on random probabilities. Those are dependent on what’s in the training data set, what questions were asked, what results are reinforced, and how it’s used. If the training data is bad and full of bad data and falsehoods, the chances of incorrect, and even dangerous, responses being generated are quite high. If the training was biased, the output is likely to be very biased. And if it’s not “trained to please”, the models are fundamentally designed to “learn to please”, so if computations that are a complete lie will increase utilization of, and faith in the model, that’s what will happen.

Moreover, every vendor is now believing the hype from the big LLM players, treating the technology as real Artificial Intelligence (when it should be called Artificial Idiocy), and trying to plug it in everywhere … promising that it will provide their clients with true natural language interfaces, agentic tech, and even BS AI Employees. Those who are adopting it are literally getting dumber by the day. Not only has the cognitive impairment, atrophy, and potential long-term decline from regular use been well documented, but the failure rate has been well documented as well with MIT and McKinsey studies demonstrating success rates of 5% and 6% successfully. Most pilots are being abandoned, sometimes before they even begin, because the tech isn’t even good enough to put into employee’s hands for the tasks the overpriced Big X consultancies claimed the LLMs would be perfect for.

Furthermore, even when organizations are smart enough to ignore Gen-AI, over-automating using the most advanced last-gen (A)RPA and AI technologies will still give Procurement teams a false sense of security and, due to their very low error rate, as time goes on, the team’s skills will go rusty and their ability to deal with true exceptions quickly disappear, especially as the old Pros (get forced to early) retire and the younglings have never dealt with exceptional situations.

The age of AI hype has ushered in a knowledge devolution faster than any age that has come before.

I hope the profession can survive it!

Why it’s just easier to do your job in the Age of AI

Every year articles make the rounds on how to do nothing at work or look busy to either slack off or keep your annoying cow-orkers away so you don’t end up having to do their work too. Now, a few in 2006 really annoyed me and I decided to point out how stupid they all were by selecting one in particular and pointing out just how stupid it was because it was harder and more work to follow the advice then to just do your job for your 35 hours a week you had to work.

But I’m not here to tackle another inane article about how to slack off or avoid responsibility, because hopefully by now you’re not dumb enough not to fall for it, especially since smart managers (and workforce monitoring solutions) aren’t either.

I’m here to tell you why it’s just easier to do your job in the age of AI than to try to use AI to do it for you.

1. You’ll have to tell the AI what you want at least 3 times. Maybe 10. Or More.

Gen-AI LLMs should be called Artificial Idiocy. There’s nothing intelligent about them. They don’t understand anything. It’s all a Grand Illusion. Automated puppet-theatre … and we’re the suckers born every minute for using it.

As a result, they produce a lot of good sounding text that sounds close to what you need, but upon further review, always misses one or more key points of your request … until you repeat it and rephrase it ten times ten different ways and your strive to perfection slowly reduces to anything acceptable whatsoever.

2. You’ll still have to edit the final outputs and execute the work in the tools it can’t drive because they don’t have the APIs for the LLM to execute.

So, after spending hours trying to get it to output the perfect report and process, you have to go edit, finish, and execute that process. Spending as much time as if you just skipped the AI in the first place.

3. You have to deal with the consequences of the errors you miss.

If you’re using a modern Procurement system which is AI-first and allows external AI to drive it, a request to “restock the supply room at the lowest cost” can result in ten times the amount of printer ink, paper, pens, and cleaning solvent you go through in a year because it hit a volume break at the production plant and skipped the intermediate supplier, ignoring the fact that you have to rent a warehouse for the truckloads of cleaning solvent you just ordered for your office building (even though you only use one floor — it assumed you wanted enough for the twenty story office building).

You’ve spent too much, have no room for the inventory, and have to explain to your boss why ordering years worth of office supplies was a good idea.

4. You have to explain the AI bill that was five fold what you planned.

While also explaining that it was your idea, not the AI’s, to screw everything up.

It’s always easier, and less risky, to use your Human Intelligence, and not the AI, to get your job done.