In a post on LinkedIn a while back, THE REVELATOR indicated that the real reason Gartner sucks (and that their stock dove 30%) is because, at the end of the day, they aren’t very good at tying advice to outcomes (and likely don’t even attempt to do it at all most of the time in ProcureTech). But in all fairness, that holds true of all the Big Analyst firms and Big X Consultancies. Also look at Forrester and IDC reports — it’s always the same old vendors or the hype of the day, whether or not that hype is delivering any value whatsoever. (And the answer is “very little” for intake and orchestration — because you can’t orchestrate an empty pit and if you attempt to orchestrate an elementary music class, be prepared for the migraine of your life — and essentially none for Gen-AI, with MIT pointing out that only 5% of deployments are delivering any value whatsoever.)

But it’s not just the Big X analyst firms. It’s the Big X consultancies as well! Now, I know you are saying “but surely they do better, they are consultants, they do projects, they have best practices, and they’re paid for results” and while that is all true,

1. they’re not all experienced consultants (and the number of juniors on many projects is scary — I’ve heard too many stories about a PE firm trotting in a McKinsey or Accenture^* after a big acquisition (because it’s their standard acquisition playbook) to optimize and rightsize operations who come in with a team of 20, of which only two actually provide value beyond what the company already knew. One of the biggest companies in our space literally marched them all out at the end of the day and told them NEVER to come back because when it came to ProcureTech expertise, they identified one individual (the project lead, who they’d likely never see again) who was sharp and got it and would definitely be able to add value if entrenched in their operation, one (his right hand man) who was smart, hardworking, and capable of learning fast and who might be able to add value, and 18 juniors who didn’t know anything that wasn’t in the 7 year old playbook on Procurement handed to them when they started, a playbook this company had rewrote multiple times over the years)
2. they don’t all have deep relevant project experience in Procurement (or whatever business function you’re bringing them in for) in your Industry
3. their “best practices” are super generic so they can be applied across the board, which means they are not tailored for your industry and definitely NOT tailored for you (so they are not best)
4. and they are paid on promises of results, which sometimes don’t materialize

Just like I keep saying it’s not the analyst firm, it’s the analyst, it’s not the consulting firm, it’s the consultant, and the sad reality is that the bigger the firm, the smaller the percentage of senior experienced talent in that talent pool, as the best talent who don’t make partner (and then have to focus more on managing and selling than project delivery) are constantly recruited by clients, consultancies, and even tech companies or the ones able to go out and join/build niche consultancies. There ends up not being enough senior, experienced, talent to go around and you’re essentially playing the lottery that one of these resources will end up full time on your project.

Since these consultancies want to be outcome focussed, in an effort to do that with more junior people, what ends up happening is they end up writing the advisory playbooks as metric focussed — what percentage of spend is on personnel in a best in class, what percentage of spend is on tech in a best in class, and what is the typical breakdown of headcount and tech spend by module or platform. Then, they tell you:

• your headcount spend is too low, so you need to go out and hire X people in Y roles because, well, metrics and statistics and that will help because of scripted reasons (more sourcing pros mean more events mean more savings, more supplier managers mean better quality, etc.)
• your headcount spend is too high, so you need to fire X people in Y groups because they must be tripping over each other and/or bringing your profit margin down
• you aren’t spending enough on tech, so go spend 10 Million on Gen-AI and that will automagically fix everything
• you are spending too much on tech, so go out to bid for a new ERP, S2P suite, orchestration platform, etc. because you obviously didn’t go to market right when you bought your current tech

Not realizing that

• the headcount needs differ in every industry AND every company
• the tech needs differ by industry, company, and process
• it’s not spend, it’s ROI per spend

and this means

• you might only need one supplier data manager in commodity indirect because there’s always three more suppliers waiting to supply you the same thing
• but you might need ten supplier relationship managers in direct, each managing a different supplier (pool) producing a different, custom, component for your advanced engineering or biomedical device
• you might not need best in class optimization backed sourcing for indirect because automated auctions will get you market price every time
• but you might need best in class optimization, analytics, and market should-cost modelling platforms to get a grip on your direct sourced custom designs
• and sometimes spending more on headcount and tech than across-the-board “average” yields a significantly better return because your quality stays high, stockouts only occur during global disruptions, your data processing is 95% automated freeing your staff to focus on strategic issues, etc.

But what can we expect from fresh grads with little mentorship (who are rushed into Gen-AI “training”) who get all of their insights from these Big Analyst firms that

• publish quadrants and waves that are completely unrelated to reality for the majority of companies with the same 10 to 20 large vendors every year that only work for select large enterprises (and the other 40 to 80 vendors continue to be completely ignored),
• constantly push and promote context-free (Gen-)AI, despite one of these firms publishing a now buried/deleted study a few years ago that stated 85% of AI projects fail and the recent MIT study that tells us, no, in fact, 95% of these projects fail to deliver any value, and
• unless you get one of the few analysts who actually gets it, employ playbook-based responses to inquiries that don’t have any context (because the analysts don’t have any time to create tailored recommendations to context because they spend too much time doing basic data collection where 80% of it could be captured in a survey monkey tool [or 95% by a well trained SLM {or, better yet, classical semantic tech with provable accuracy rates} that could map free text to standard process needs and vendor solution categories for easy verification and correction by a true human expert]).

The reality is that until

• big analyst firms and big consultancies admit their flaws,
• start tying actual outcomes to the standard projects/recommendations they made, and
• start analyzing and using these results to tailor recommendations to their clients that have a good chance at actually delivering value

these firms, and their standard recommendations, are going to continue to suck and your chances of success are going to remain at 12% for standard projects and 5% for Gen-AI projects.

Sad, but true.

^* not realizing that the reason the company was such an attractive acquisition target in FinTech/ProcureTech was because they already knew all the best practices that the big firms have in their playbooks and were employing them effectively; these Big X tend to do well on average companies that are not best in class or deep in modern process or technology

A recent post on LinkedIn noted how there’s a wave of AI products flooding the consultancy and advisory space and how they are, frankly mediocre, overpriced wrappers on public models with minimum innovation, if any.

This is sad, but true, and it’s not the worst of it. The worst of it is that some of the Big X firms are training tens of thousands of consultants and f6ckw@ds on these tools to generate hundred page pitch decks and three hundred page strategy and implementation guides of standard generic, meaningless, drivel to deliver to you as “highly tailored guidance and expertise from their leading partners with 20 years experience delivering high-value projects” and charge you tens of thousands of dollars for the privilege.

This is especially egregious when you can use free/cheap (and I’m talking put it on your personal credit card cheap because you won’t notice the fee that is less than your monthly coffee charge from the coffee shop) to build the exact same pitches, strategy, and implementation guides from the thousands of freely available documents on the web in a few hours with a few generic prompts over a Sunday morning coffee. (And then, when the coffee kicks in, realize it’s all a load of cr@p and put in the bit bucket, but at least you will know what a load of cr@p looks like in pitch deck, strategy guide, and implementation plan form and will recognize it the next time an overpriced Big X tries to sell it to you for a ridiculous price tag and will have learned something from the exercise.)

Now that there are companies selling overpriced “custom” products to these consultancies, the situation is only getting worse, especially when the “customization” is just a wrapper with some pre-engineered prompts that aren’t well tested, only work at a point in time, don’t really give the consultancies what they need, and sometimes translate mediocre inputs to inputs that are even worse. Moreover, when you consider the price is sometimes a 100X multiple on the products they build on top of, it’s disgusting. Consultancies are paying more for less, and, in return, you are paying even more for even less!

Which makes no sense when the current publicly available LLM tech is being offered cheap (to try and hook you on it, even though, as we’ve repeatedly explained, the tech is not ready for prime time and will never deliver more than a fraction of what they are promising), and new implementations will get a lot cheaper. Just look at how DeepSeek undercuts the cost by a factor of 100 and gets 90% of ChatGPT (as long as you don’t mind exposing all of your secrets to the CCP). LLMs are nothing more than a fancy next-gen “deep learning” Neural Networks that construct responses vs. serving up canned responses (which is why hallucinations and lies are a core function, not an error that can be trained out) which gets us closer (but no cigar) to decent natural language processing (NLP) for the express purpose of the generation of desired outputs from inputs, but not there (and now, in addition to all the false positives and false negatives, we had to deal with, we now get to deal with hallucinations and lies as well). It’s not secret magic, it’s layers and layers of interconnected statistics and probabilities that no human can understand, in rather standard models that any Theoretical CS and Applied Math PhDs can build, and implementations that are better and cheaper are going to keep appearing as time goes on.

This means three things to any consultancy thinking about using these custom “AI” solutions

• you still have to be even more tech savvy to use them to any degree of effectiveness
• it’s not “the art of the prompt“, it’s the art of the training (even though they don’t really learn because they are NOT intelligent) because that determines the maximum level of effectiveness you will ever reach with them (and you need to provide them with sufficient correct data, which needs to be in the high gigabytes at a minimum, and, preferably, in the petabytes)
• you don’t have to worry about when they are right (enough), which will happen between 90% and 95% of the time with proper training and proper prompting, or when they are obviously wrong, which will happen a very low percentage of the time (say 5% to 9%), but when they are oh so wrong but the response is constructed in a way that is oh so convincing that an above average person in intellect and experience wouldn’t know otherwise (that danger zone between obviously wrong and good enough that is likely only 1% to 2% of the time).

Now remember that your consultants aren’t that tech savvy, and you should know right off the bat incorporating and using these is going to be difficult and time consuming. (There’s a reason we are constantly advising you to be very careful about using Big X for tech selection and tech projects, and that’s because, even though they say it is, it’s NOT their forte. They weren’t built on tech, and they don’t have the best talent in tech — that talent goes to the big tech companies who can offer the 500K salaries to leading devs or the wild-west startups that leading devs think are cool.)

You only have so much clean and complete data you can use for training. You can’t just throw in the 1000s of decks you’ve built as you can’t share work you’ve explicitly created and sold to past clients, and the AI won’t anonymize the decks and suggestions (even though you think it will). It won’t know that “Ford” is the name of your client and might think that “Ford Data” is another term for shallow data and copy sections from that custom strategy straight into your pitch deck for General Motors (and chances are your overworked junior consultant won’t catch it when skimming that 200 page deck with only 2 hours to go before the meeting). And we know what happens then … (and it ends with the consultancy not keeping either client).

It will take a lot of analysis to identify those 1% to 2% of cases where it is very, very wrong but so convincingly right that you will miss some. What happens when you do and give your client advice that explodes in their faces? (We’ll let you answer that one.)

And for you as a consumer, if your consultancy is using this Bogus AI tech, it means that:

• the situation that results from solution delivered might be even worse than the situation you started with (as should be evidenced not just by the tech project failure rate that is approaching 92% but the fact that 42% of projects are being abandoned during implementation!)

A solution designed by Gen-AI is not a solution. A real solution is a solution designed by human intelligence that uses real, augmented intelligence, to research and validate that solution. Remember that if you are going to hire a consultant!