Category Archives: Market Intelligence

Operations Research Lessons are Timeless … Especially in the Age of AI

Twenty years ago we published a piece focussed on Mark Daskin’s Everyday Lessons from Operations Research that he delivered at INFORMS 2006. They are just as relevant today, if not more so, in the age of AI (Hype).

  1. Service and Performance Gets Worse as Utilization and Variability Increase
    This is true in so many ways. Dumb users asking stupid questions with poorly formed prompts that don’t match the training prompts flooding the internet with poor and hallucinatory answers that are fed into the next AI model in the collective race to the bottom. Response times decreasing as the models need to get bigger to process ever more mediocre data and, hopefully, create better responses (even though that won’t happen). Costs going through the roof as a result of the constant need for more computing power.
  2. … But Variability is necessary.
    Because every request is fundamentally different, even if phrased the same as the lack of standardization, and training on that standardization, ensures every user has a different intent with their request.
  3. Expect the unexpected in today’s world.
    Most responses will be okay, some will be bad, a good number will be completely hallucinatory, and every now and again the hallucination will provide a revolution insight — which will go unnoticed because most users now lack the cognitive power, and possibly the intelligence, to even detect a hallucination in the first place due to the mass cognitive atrophy Gen-AI is, provably, inducing.
  4. If it is too good to be true, it probably is not.
    But yet we continue to believe the lies, damn lies, and hallucinatory hype spewed by the purveyors and pushers of Gen-AI and the garbage it spits out.
  5. Life is full of errors.
    And Gen-AI multiplies those errors tenfold to ten-thousandfold!
  6. A good decision can result in a bad outcome.
    Even when the AI gets it right on the data, the data is always backward looking, not forward looking, and the right decision based on a past trend can still be wrong if a key variable or assumption just changed and the data does not yet reflect that. So even when the AI actually gets it right, it can still be wrong!
  7. If you are not using all you have, don’t pay for additional quantity.
    If you only have a small amount of data to train the model on, don’t pay for the largest model available — not only will it not help your case, it will hurt as there won’t be enough data to “train” the model even to reasonable levels of accuracy (relative to what the model is theoretically capable of). Always go with the smallest model that will theoretically meet your needs — and, moreover, if you have multiple distinct needs, multiple distinct models will perform much better than trying to create one big mega model.
  8. You can never do better by adding a constraint.
    Never, ever, ever. Whatever you are trying to minimize will increase. Constraints restrict options and that will always result in increased cost, increased risk, etc.
  9. Keep it Simple
    The best AI, especially in the age of Gen-AI, is no AI. More specifically, don’t use AI where AI is not needed. Deterministic automation, when properly applied, has worked well for decades and now, in an age where we can encode adaptive workflows, suggest exception resolutions, have the rules automatically update when a resolution is accepted or created, and improve over time.
  10. Think about problem formulation.
    The formulation dictates the potential solutions … and the better you define a problem, and a stepwise algorithm to resolve that problem, the less need you will actually have for (Gen-) AI.
  11. Look for compromise solutions.
    If a problem ends up being defined particularly constrained or complicated when multiple parties get involved, work together to simplify the constraints or complexity where it truly isn’t needed. There’s a big difference between preference and requirement, and understanding that allows for the creation of good comprise models and solutions.
  12. Data is not information.
    It never has been, and never will be. Moreover, data pulled out of an LLM might not even be accurate, so the chance of it producing usable information is even less. Always remember that.

In the Age of AI, the Analyst is King!

Never forget this!

(Click here to learn why DPGLabs.ai is best tool a VENDOR has to judge their perceived place in the ProcureTech market if that’s what you’re looking for.)

This rant is inspired by Dr. Elouise Epstein who, in this LinkedIn post, said the most wrong and dangerous thing she ever said. (And this SI article is an archive of my immediate rebuttal, found in this LinkedIn post.)

Saying that in “in the Age of AI, analysts are superfluous” is akin to saying that, “in the Age of the Internet, the Journalist is unnecessary” … and we all know how false that was, and still is. (Because the claim that the internet allowed everyone to be a journalist was not only far from the truth, but also ignored the fact that many people misuse the internet to spread rumours, conspiracy theories, and false facts that, when combined with FUD (fear, uncertainty, doubt), caused others to spread and amplify those same false facts. And if Gen-AI has proven one thing, it’s that it can spread and amplify falsehoods at the speed of light.)

(Now, this is assuming Dr. Epstein believes that an analyst does more than sticking random logos on a worthless map. But if that’s her definition of analyst, then maybe she’s right.)

There are multiple problems with her statement, and more problems than can fit in the 3K post character limit of LinkedIn, so we focussed on the biggies, as that should be more than enough to point out how Gen-AI has made the Analyst more critical, not less.

1) We are not in an age of true AI. We are in an age of hallucinatory Gen-AI.

This means that, even if all the data available is mostly correct, the AI can still get it wrong.

2) DPG Labs is NOT using evidence, they are using content from various URLs and weighting it (and, if the weighting is high enough, calling it evidence).

This means that if a vendor floods third party sites with unmarked advertorials filled with fake or misleading content, the vendor profile that Gen-AI creates is sure to be wrong.

3) Most vendors and customers blast success stories and hide failures.
(No one wants to admit they blew millions of dollars.)

These only come out in private conversations and the only “evidence” comes a few years when the customer switches to a different platform much sooner than the norm.

etc.

This means that no auto-generated profile can be considered reliable without human review. (And the odds of correctness range from 95%+ for the major vendors where there is lots of correct content to work on to 50%- for new vendors with little content or a lot of misleading/inaccurate content.)

But even if the profile is mostly right, tech cannot do the following:

4) Judge organizational fit.

That goes beyond a score. (That’s why Xavier Olivera and I designed the SpendMatters, now Hackett, Solution Map as a TechMap. We assessed what most organizations couldn’t, gave them average unbiased customer scores against that assessment, and let the organizations focus on what they could assess — their needs, the vendor’s culture, service, relationship approach, etc. — and not what a static map can’t do.)

5) Give you a true timeline and success rate.

Vendors over promise and under deliver, as they all assume perfect scenarios that never materialize. Dumb Gen-AI can just assess statements, not fact, and can’t understand what factors are involved in the 3 month best-case implementation, the typical 6 month implementation, and worst-case 12 month implementation for organizations of similar size, complexity, and geography in the vendor portfolio (where even the best case 3 month is still twice as long as what they promise).

6) Help you decide if you can trust the vendor.

This is probably the most important point. Do they have what they claim, can they deliver it, and are they committed to delivering it to you on time, schedule, and budget. And it’s not just their word, or a third party survey on a select customer sub-set, it’s based on realistic assessments of performance and the individuals assigned to your project.

etc.

That being said, DPGLabs.ai is the best tool a VENDOR has to judge their perceived place in the market.

It’s also the best foundation out there on which to build a true next-generation analyst offering. If this existed a decade ago, the efficiency gains it would have enabled would have allowed us to cover 80%+ of the vendors in depth at Spend Matters (vs 20%-) on Solution Map as it would have slashed our research and writing time* and allowed us to focus on verification, gaps, and uniqueness.

I described why the offering is the best tool VENDORS have, as well as how it could be the best tool to build a new analyst offering in the comments (of the original post) in detail in this LinkedIn post, where I have included the content below for easy access.

But even though the offering has a lot of value, it’s still nowhere close to a replacement for a true analyst.

The true value of DPG Labs.

After 20 years as an analyst, I’ve learned a lot.

This includes the fact that a lot of vendors don’t know how to position themselves, and know even less about how they are positioned in the market.

To this day, I still see new vendors regularly misusing purchasing, procurement, and sourcing when describing their offerings and wondering why they aren’t showing up in the right lists, maps, or articles.

I see vendors over-focused on the hype-du-jour (AI, Agentic, Intake to / Orchestration, Analytics, etc.) who completely ignore descriptions of rock solid capabilities or unique offerings that would differentiate them from the 40 to 100 other offerings that sound almost the same. (See the SI Mega Map if you don’t believe how many competitors you really have! The 7EB V1 edition has 888 vendor logos.)

I see vendors who have no idea how the market, and more importantly, how the search engines / AI tools used by the market see and position them.

What David Bush and his team have built at DPG labs is the perfect answer to that. By correlating and cross-referencing to a common vocabulary all of the external data and coverage on you against a standard functional map definition (that captures the questions buyers are likely to ask), you can see which categories / verticals / etc. the market is putting you in and which vendors you are being put up against. If those aren’t the vendors you expect to / want to be put up against, this means one or more of the following is true:

  • you’re not describing your offering appropriately
  • the coverage you’re getting is not focussing on the key / unique / valuable aspects of your offering (and you have to change your analyst relations / marketing / partnership approach)
  • you’re (trying to) compete in the wrong market

and you have rock solid proof you can take to the C-Suite that you need to market / sell / partner different if you want to succeed.

Up until now, analysts could only tell you this … and you/the C-Suite could ignore us (by claiming it was our opinion — which it was, but our expertise meant we were right the vast majority of the time). But this tool is essentially doing what pros using AI (enhanced) tools are doing to research you, and showing you where you stand.

It’s the one offering that I would have loved to create when I was at Spend Matters, but never could (because, even 3 years ago, the AI models just weren’t there to support it). And it’s the one tool you need the most. (Plus, market clarity will ensure you score better on the right maps since the analysts will have a better understanding of you before the first interaction, which will help them ask the right questions, focus the demos, and give you the right coverage.)

And even though you can use it to create a shortlist as a potential customer, you can’t use it for final selection. I explained why in this comment, where I also reiterated how great it would be as the foundation for a next-gen analyst firm. The core rationale is the following.

We have to remember:

  1. publication is NOT evidence, it could be opinion
  2. probabilistic assessment of language is not human assessment of language
  3. news coverage is often still marketing, especially when it’s an advertorial or there is paid coverage
  4. disgruntled customers can spread lies as well as truth
  5. even “verified” sources can be hacked or altered or replaced on the internet
  6. … and when (correct) coverage is insufficient, errors (and hallucinations) multiply in Gen-AI (frontier models)

Your (Gen-) AI tools don’t know any of this! Unless every profile you build is reviewed by an expert human AFTER an expert human review of the solution (i.e. a real analyst with real knowledge and experience), which DPGLabs is not doing (as they don’t have a real analyst team), you can’t be sure it’s accurate and not free of hallucinations.

P.S. This means the only profiles I’d trust on DPGLabs are the human reviewed profiles that Dr. Elouise Epstein reviewed, which would be the profiles of the vendors she knows well, and that’s a small minority!

* Not that Gen-AI produces good content, but, as per above, when enough material exists, it can create great summaries that allow an analyst to get a reasonable expectation of a vendor offering before the demo, produce summaries for the vendor to self-correct, and produce the auto-gen fact sheets, allow the analyst to focus their efforts and writings on the true analysis, opinions, and insights that you really need.

Have We Lost The Economy of Information?

Twenty years ago, we were forming GPOs and Purchasing Consortiums to take advantage of Economies of Information. Ten years ago, we were not only rolling out suites that focussed heavily on consolidating spend (and performance) data globally (with the likes of Coupa and Sievo boasting about how much spend they had managed and normalized) to take advantage of the economy of information global spend data gave us, but building best in class analytics solutions to take advantage of all of the data they (could) gather(ed).

In the age of (predictive) analytics, which preceded the current age of AI Hype, the importance of data finally started to become recognized and you had a number of startups hit the scene providing next-gen data feeds. (Near) Real-time commodity indices, market price data, risk data, company financial data, carbon data, energy rates, water rates, regional overheads, average process time, average GPO and transaction rates, average performance data, etc. Any organization that wanted to build a best-in-class should cost model, best in class performance model, etc. The information was available, there was an economy for information, and the economy of information was right around the corner.

Let’s step back and define what we meant by this. On economies of information, twenty years ago we wrote:

The consortium of the future offers the benefit of expertise more so than it offers the benefit of scale. Eventually, especially with constantly rising raw material prices, the best practices employed by a competent consortium will squeeze all of the fat out of the supplier’s margins and the best price will be obtained. Once this occurs, the consortium will use its expertise to assist its members in advancing purchasing technology, reducing wasteful consumption, and improving the application of the goods and services they purchase. Since a consortium has access to all of the knowledge of its members, it can tap this knowledge to identify the best potential suppliers with the best potential products and services to meet member needs. Furthermore, this gives it a much better chance of identifying and qualifying low risk suppliers.

In other words, with a fact-based outlook on reality, consortiums could help take Procurement to the next level. Then, when the data-stream startups made all of that same information easily available as plug and play data feeds into your suite through standard APIs, the true economy of information hit Procurement for those who wanted it and Procurement could make insight-based and fact-based decisions and get better.

But now that we’re a few years into the age of AI Hype, I believe we’ve lost the economy of information. There a few reasons for this:

  • we’ve replaced data feeds with LLM chatbots like clod and chat, j’ai pété and assume they have access to the same data, and, most importantly, the same ability to run predictive analytics on that data
  • despite claims to the contrary, the LLMs are getting worse by the day … now that the majority of data on the internet is AI generated slop, being cross fed into other LLMs, regurgitated with compounding errors, we are not only losing the core data in the tsunami of slop but the meaning of that data as well
  • with LLMs being cheaper than data feeds, the data feeds have been ignored, a number have went out of business, and the rest are floundering

There’s no information without actual, verified, facts and intelligent interpretation, and the majority of that has been lost in the age of AI hype.

If too many real data providers, as well as applications that deterministically and intelligently integrate and analyze real data and real facts, go out of business, there will be no solid foundations for real information, and, thus, no solid foundations for economies of information — and then we’ll be back to the Procurement dark ages.

Technology has never advanced Procurement. Only facts, data, process and decision improvement based on intelligent interpretation has.

It’s Time To Put An End to Business Spend Management!

Coupa may have built their Billion dollar business on it, but the time for Business Spend Management (BSM) is at an end!

Spend Management may have been the business strategy and philosophy that leaders practice and followers fail to understand according to THE PROPHET (who penned it two decades ago), but that was then, and this is now.

While spend management wasn’t supposed to be just cost management, but instead an incremental change that represents a new type of thinking, a way of taking integrated approaches to not just procurement, but all aspects of non-revenue generating operations and a way of thinking about your global supply chain strategy that would reduce costs, improve processes, and increase profits, that’s not what it became.

Spend Management became a fancy term for cost reduction (and, for the most part, that didn’t include cost avoidance), and even when optimization was used, the objective was always price. The majority of analytics focussed on spend, and even when process analytics were run, they were always framed in terms of cost and savings if automation was increased and/or process time decreased. Even multi-objective RFPs were primarily weighted on price, if they were weighted on anything else at all once basic supplier, product, and/or service requirements were met.

And now, in the age of AI (hype), all of the agentic / BS AI Employee offerings are focussed on offering you solutions to your Procurement problems that will reduce overall cost (base cost, human cost, processing cost, etc.). Cost, cost, cost.

But, for as long as it has existed, that’s never what Procurement was about. The goal might be to keep costs down, but the point of Procurement is the acquisition of a good or service. Supply, NOT Spend. Now, it’s true that the spend has to be less than what an individual would pay on his or her own to acquire the product or service or, as per Coase, there’s no reason for the business to exist, but if the business can’t acquire any products or service to allow them to offer a product or service for sale, the spend doesn’t matter.

That’s why BSM (where the “B” doesn’t necessarily mean “Business”) needs to end. It’s time to return to supply management, with guarantee of supply (not minimization of spend) at the forefront. This means a focus on risk minimization, production management, logistics management, trade management, and other variables that contribute to assurance of supply (and prevention of disruption). Costs can only be optimized once all of these other factors are taken care of.

This is the only way to procure in today’s volatile times, and, moreover, the only way to control cost. Real spend management is not just minimizing unit costs, transportation costs, and other purchase prices, but reducing operational costs across the board. You’re not reducing costs if the organization has to buy 30% off of contract because of delays, disruptions, and defects. And you’re definitely not reducing costs if you have to constantly expedite shipments, buy from alternate sources of supply at higher prices, or lose considerable revenue due to stock outs while paying for warehouse space / retail space.

It’s optimizing human intelligence against dumb automation to make sure processes are optimized, exceptions are quickly processed, risks mitigated to the extent possible, and disruptions detected and addressed as soon as they arise.

It’s Supply Management. (Not Spend Management.)