Category Archives: 101 Damnations

Firms that Rely on Logo Maps and Analyst 2*2s for Tech Selection are NOT Appropriate for Tech Selection!

In our last article, where we described in detail the many, many reasons why logo maps (including the Sourcing Innovation Mega Map on Source to Pay+ with 666 Unique Clickable Vendor Logos which were verified to be valid as of 2024 April 13), we not only reiterated how these maps are mostly useless but explained that your mileage will vary widely between a map created by an analyst who’s likely seen 1/3 to 1/2 of the vendors in depth and a(n) (former) implementation consultant or (want-to-be) influencer from a CPO background who has no in-depth technology education or experience (beyond the systems he used).

Those who read between the lines would have seen this post coming — not only are they not appropriate for tech selection, any firm that relies solely on them or analyst firm 2*2s (which are great if you are searching for some holy smoke to keep the beast of procurement technology at bay) is also inappropriate for tech selection projects.

Your results with such firms will be about the same as the bigger firms with “consulting partner” status with all the (same) big players, as they will ultimately just recommend the same ten firms for your Tech RFP over and over again, whether or not they are the right firms (and solutions) to meet your needs.

In order to effectively select a set of potential solutions for a client, you need to, at the very least:

  • understand the processes the client needs to support and the gaps they have
  • understand the solution types needed to support the processes, and the client’s gaps in particular
  • understand the client’s current technology landscape and Technology IQ, including what is replaceable and what is not (since, gosh darn it, some clients are going to hold onto that ERP they overpaid for until you dodge their six-gun pistols and pry the contract from their cold, dead hands)
  • understand the client’s unique situation based on vertical/industry, market size, and geography/culture
  • understand what global vendors support the processes, fill the gaps, synch with the tech stack, and can, possibly through third party integrations/partners, address the client’s unique requirements

This is a tall order. So tall in fact that, despite the growing demand for technology transformation and digitization across the Procurement landscape, outside of a few niche vendors that primarily focus on specific industries and specific solution types, the vast majority of procurement transformation shops aren’t able to fulfill it. Most will

  • have the processes down pat, they are consultants after all!
  • have a decent understanding of the common/core solution types, as they smart ones will actually read the expository articles written by the analysts (that they have access to anyway*)

Some, who employ technology and industry-specific professionals, will be able to build a decent understanding of

  • the client’s technology landscape and technology quotient
  • the unique requirements to look for/enable based on vertical/industry, organizational size, and geography

But few, if any will be able to:

  • identify even a handful of relevant global vendors that take into account the first four requirements

This is because, as pointed out in our last few articles:

  • the space is much bigger than they think, with
    • more types of product offerings,
    • considerably more vendors then they think exist, and
    • considerably more than they can process
  • they don’t have the deep technical background or technical understanding to differentiate between two vendors that speak the same and present applications that look the same in a 60 minute demo, but differ greatly in underlying power, extensibility, integration capability, etc. where you need a deep technical background and/or competitor understanding to tease it out (as well as a deep understanding of Procurement and the competitive [solution] market place)
  • they don’t have a process to do a proper technical assessment, diligence, or tech analysis …
  • and they certainly don’t know how to do a deep assessment by module/area to truly differentiate two solutions to qualify them as suitable for selection if they submit the best RFP

As a result, many consultancies will just do their in-depth process analysis, write up functional requirements based on that, and toss it over the wall to the solution providers to figure out, selecting from their partners if they feel there is enough overlap, then from the upper right in the analyst maps they paid for, and, finally, from the logo maps from their most trusted source. And, as we’ve explained, this doesn’t cut it and is why many sourcing / procurement software selection projects fail to live up to client expectations. Because, and we can’t say this enough, the most you can use logo maps / analyst 2*2s for is vendor discovery. Not validation for your projects!

Now, while the doctor has yet to receive an answer to his transformation process inquiries from any consultancy/service provider that fully satisfies him (he is demanding, after all), he is happy to say that, recently, a few# providers have acknowledged that transformation is going to require getting a lot more intelligent in tech and updating their processes and methodologies to recognize that, while it’s still The Wild West, it won’t be tamed by hope and grit alone — you’ll need the right tools to conquer it (and, FYI, those tools aren’t Gen-AI, they are good old-fashioned predictable, dependable steam- and gunpowder-powered tech solutions in the hands of us old and busted masters; the new hotness has nothing on us).

Secure Download the PDF!  (or, use HTTP) [HTML]
(5.3M; Note that the Free Adobe Reader might choke on it; Preview on Mac or a Pro PDF application on Windows will work just fine)

 

* this is your regular reminder that Sourcing Innovation has never had a paywall and never will for baseline vendor coverage or expository posts; should SI choose to offer books, in-depth [comparative/market] intelligence, or similar IP services, for example, it may in the future sell this non-blog content, but every blog post will remain paywall free — almost 6,000 and counting …

# and we mean few, he can currently count them on his fingers on one hand, thumb not required

Like Analyst Firm 2*2s, Random Logo Maps are NOT Appropriate for Tech Selection!

In our last article, we explained why the doctor created the The Sourcing Innovation Source-to-Pay+ Mega Map, even though he despises logo maps. It was literally the only way he could expose how every one of logo maps released to date was completely useless (and some to the point that they were harmful, but that’s another rant for a later time).

In a nutshell, all of these maps had the following problems, which were correctable (and corrected in the Sourcing Innovation Mega-Map):

  • vendors / solutions no longer existed as of release date
  • categories were meaningless and not actual solution modules
  • vendor logos were not clickable or even footnoted (so you had no clue what that ruin actually represented — a new age vendor or a demonic symbol from a long lost hieroglyphic or symbolic language)

As well as the following problems, which still exist in the Sourcing Innovation Mega-Map (SIMM) because some of them are just not (fully) addressable:

  • nowhere near complete (the further you get away from the Source-to-Pay core, the less complete the SIMM likely is
  • no indication that the landscape changes DAILY (vendors come, get acquired/merge, go out of business, add new capabilities and modules, drop existing modules, etc.)
  • the vendors aren’t always comparable even at a functionality baseline
  • not all vendors with a comparable solution are relevant to the same (type of) company

We ended our last article noting that the right vendor for you was dependent not just on the module(s) you needed to address the process gaps the transformation consultants defined, but the industry/vertical you are in, the size of your organization, the cross-organization user base you need to support, and their technical intelligence (TQ).

The logo maps don’t capture that. (But, to be fair, the vast majority of the analyst market maps don’t either.)

But more importantly, they don’t always (accurately) capture what solution(s) a vendor accurately captures. The reason for this is that, to be completely accurate, the creator would have to be fully aware of, and have seen, the current full end-to-end solution as of the day the map was released and then accurately map the providers’ solutions to each of their logo map categories.

the doctor has likely reviewed more Source-to-Pay solutions over the past 19 years as an analyst than almost any other analyst except for Mickey North Rizza (15 years at AMR / Gartner / IDC), Duncan Jones (who was Forrester Vice President and Principal S2P analyst for 16 years straight), and Pierre Mitchell (25 years at AMR / Hackett / Spend Matters). (Just about every other technology analyst still active in our space has only been a full time market analyst for a decade or less.) Even though he has reviewed, in depth, over 500 hundred solutions (and written about 350+ in detail on Sourcing Innovation and Spend Matters [but good luck finding at least 1/3 of the Spend Matters coverage since, as previously mentioned, the site refresh dropped co-authors on many articles, many of his articles were co-authored, and he was always second billing], he hasn’t even seen half the solutions on the map in depth (but still believes the ratio of in-depth vendor knowledge that went into this map is still greater than every other logo map produced). As a result, his classifications, like any other analyst, are based on, in order:

  • in depth demos, diligence or evaluation projects
  • detailed vendor communications (beyond just what’s on the website)
  • website / third party summaries (looking at the functionality where possible, not just the language)

Which means that, if the website/materials were out of date when reviewed (and it’s been a few years since the last review), the classification could now be highly inaccurate. The eProcurement vendor could have shifted mostly to I2P/AP. The supplier management vendor found a niche in risk management or category sourcing and dropped most SXM capabilities. The contract management solution, interchangeable with forty others, didn’t get traction and was dropped. And so on.

So if your mileage varies in this map, put together by someone who has consistently been at least a part time (if not full time) analyst since Sourcing Innovation was started in 2006, imagine how much it will vary in a map put together by a former consultant, who might have only seen the same ten solutions he was always implementing in depth, or a former product manager who doesn’t have a solid technical background and can’t accurately judge the true capabilities and potential of the solution he’s looking at. (the doctor has an earned PhD in computer science and has been a software architect / research scientist / CTO, compared to the average analyst who, in the early days, came from an operations / logistics / management background if you were lucky and a journalism,
English, or history background [because they could actually write] if you weren’t.)

In other words, no matter how cool these (logo) maps look, at best they will be mostly useless (if they give you clickable logos so you can begin your own research effort), or completely useless otherwise.

Secure Download the PDF!  (or, use HTTP) [HTML]
(5.3M; Note that the Free Adobe Reader might choke on it; Preview on Mac or a Pro PDF application on Windows will work just fine)

the doctor dislikes logo maps! So why did he create one?

To demonstrate how, to date, they have all been completely useless, with some to the point of being actually harmful, but now that the gauntlet has been cast, he expects the next version of at least one of these maps to only be mostly useless (and maybe even only moderately useless) and mostly harmless. It’s the same reason he developed the initial versions of Solution Map*, because he found all of the big analyst firm maps mostly useless, and completely useless for tech selection.

(On the tech map front, how can you compare the technical capabilities of a solution where the axis are each on subjective classifications such as “strength and “strategy” or “execution” and “vision”, and, furthermore, where each of these nebulous concepts is made up of half a dozen subjective ratings meshed into one. While not perfect, at least Solution Map gave you an apples-to-apples pure objective technology rating (as each question had a defined rating scale based on technical maturity) against an unbiased pure customer opinion. So you at least knew whether or not

  1. the vendor actually offers a readily available solution of that type
  2. how it compares to the market average of vendors with actual available solutions of that type)

Thus, if you insisted on using logo maps, he at least wanted to make sure there was at least some redeeming qualities.  However, as he has already stated, his map is mostly useless and while a few flaws were corrected on release, some are inherently not addressable.  The problem with these maps in general is that, in addition to all the weaknesses the doctor addressed in his release post, namely:

  • Some vendors/solutions no longer existed as of release date (which was addressed)
  • Many of the categories are meaningless and not actual solution modules (which he corrected, but this means the fit varies across vendors in a category)
  • Vendor logos were not clickable, and not even footnoted when all you got was some strange symbol that looks like it should be carved on a 3000 year old ruin (which is the primary improvement, all logos are clickable and take you to the vendor site as of the release date).

4. They are nowhere near complete.
Most of these maps are in the 100 to 150 logo range. As the doctor has clearly demonstrated that’s only 1/7 to 1/10 of the number of vendors in the core space. Furthermore, even though the doctor does a full database update at least annually, he will guarantee that not even his map is close to complete. While he’d wager he has 90% of the vendors actively selling in North America and Western Europe in the core Source-to-Pay buckets, that percentage goes down as you venture out into the periphery. Plus, in some areas, like ESG/Carbon, he tracks only those focussed on carbon/scope 3 accounting with supplier management / sourcing integration capability, and ignores the remaining ESG/Sustainability/Climate vendors, of which there is likely 10 times as many right now (although we’ll see a lot get swallowed up or die off as the space matures). Most of the supply chain risk vendors are missing unless they offer core supplier management capabilities, or integrate with supplier management modules, as well. And so on.

5. The landscape changes daily.
the doctor did a full database review last year when he did his 39 steps … err … 39 clues … err … 39 part Source-to-Pay+ series, and since then, over half a dozen vendors/offerings are completely gone and over a dozen acquired and swallowed into larger vendors. One, acquired in 2022 that was still offered as a standalone solution late March disappeared by the final link checks that began on April 13. So, while these maps are distributed by their creators for months, and sometimes a year, they are only valid as of the last date where the creator actually re-verified every single vendor.

6. The vendors are only comparable at the baseline, IF they are comparable at all.
If no two (2) vendors are created equal, imagine how different twenty (20) are, or one hundred (100)! If you refer back to our previously referenced 39 part Source-to-Pay+ series,

  • sourcing vendors break down into RFX, Auction, optimization and may/not contain (best-practice) templates or category expertise
  • contract management generally breaks down into negotiation support, (post-signing) lifecycle (execution) management and tracking, and analytics
  • spend analysis is similar, but differs on DIY vs. services led, load/classification support vs. self load/(re)class, out of the box report templates, autonomous analysis and opportunity identification, etc.
  • supplier management was broken down into the 10-segment CORNED QUIP mash, which expressly excluded DEI, because most application thereof is definitely NOT equitable (as the biggest promoters clearly never looked up what the words actually mean in a dictionary)
  • eProcurement, while it revolves around a PO (and, hopefully, a no PO, no pay policy), may or may not have punchout/internal/managed catalog support, may or may not support receiving, may or may not support price tiers and discounts, etc.
  • I2P, while it revolves around the invoice, it may or may not support anything beyond internal PO flip or XML, may or may not support m-way match, may or may not integrate with a payment system, etc.
  • and the same variation exists across every other category

This is assuming that the creator actually understood what every vendor offered and classified according to what the vendor’s product actually did vs. what language the vendor chose to use to describe their product.

7. Even all the vendors with comparable solutions are NOT relevant for you.

When you are considering a vendor, at the very least you have to consider

  • the verticals/industries their solution was designed on, and designed for
  • the organizational size they were developed for

and a host of other considerations based on your industry, your organizational size, and the hole you are trying to fill.

This is why so many Source-to-Pay+ selection projects end up not (fully) delivering and why most big consultancies just keep recommending the same-old same-old five (5) (big) vendors regardless of what your needs are, because they don’t know any different and at least those vendors will be around tomorrow. And this leads into a bigger discussion of why these logo maps, like most analyst maps, are NOT appropriate for transformation projects. Which we’ll take up in our next article / rant.

Secure Download the PDF!  (or, use HTTP) [HTML]
(5.3M; Note that the Free Adobe Reader might choke on it; Preview on Mac or a Pro PDF application on Windows will work just fine)

* and the doctor would like to make it very clear he had NOTHING to do with the current interface and presentation of Solution Map; it’s likely many of the questions are still his, but to be valuable, SolutionMap has to be properly scored and the ratings properly compared and applied relative to a number of factors not explicitly captured in the map

The Sourcing Innovation Source-to-Pay+ Mega Map!

Now slightly less useless than every other logo map that clogs your feeds!

1. Every vendor verified to still be operating as of 4 days ago!
Compare that to the maps that often have vendors / solutions that haven’t been in business / operating as a standalone entity in months on the day of release! (Or “best-of” lists that sometimes have vendors that haven’t existed in 4 years! the doctor has seen both — this year!)

2. Every vendor logo is clickable!
the doctor doesn’t know about you, but he finds it incredibly useless when all you get is a strange symbol with no explanation or a font so small that you would need an electron microscope to read it. So, to fix that, every logo is clickable so you can go to the site and at least figure out who the vendor is.

3. Every vendor is mapped to the closest standard category/categories!
Furthermore, every category has the standard definitions used by Sourcing Innovation and Spend Matters!
the doctor can’t make sense of random categories like “specialists” or “collaborative” or “innovative“, despises when maps follow this new age analyst/consultancy award trend and give you labels you just can’t use, and gets red in the face when two very distinct categories (like e-Sourcing and Marketplaces or Expenses and AP are merged into one). Now, the doctor will also readily admit that this means that not all vendors in a category are necessarily comparable on an apples-to-apples basis, but that was never the case anyway as most solutions in a category break down into subcategories and, for example, in Supplier Management (SXM) alone, you have a CORNED QUIP mash of solutions that could be focused on just a small subset of the (at least) ten different (primary) capabilities. (See the link on the sidebar that takes you to a post that indexes 90+ Supplier Management vendors across 10 key capabilities.)

Secure Download the PDF!  (or, use HTTP) [HTML]
(5.3M; Note that the Free Adobe Reader might choke on it; Preview on Mac or a Pro PDF application on Windows will work just fine)

AI: Applied Indirection, Artificial Idiocy, & Automated Incompetence … The April Fools Joke Vendors are Playing on You Year Round!

So on the one day of the year when they should be making the joke, I’m going to reveal it.

The vast majority of vendors who claim “AI”, where they want you to think “AI” stands for Artificial Intelligence, have no “AI” in that context, and many don’t even have anything close. A few may have “Assisted Intelligence” (Level 1) and even fewer still may have “Augmented Intelligence” (Level 2), but “Analytical (Cognitive) Intelligence” (Level 3)? Forget it! And as for, Level 4, “Autonomous Intelligence”, which is the baseline that must be met before you could even consider a system true “AI”, doesn’t exist (at least as far as we know). (ChatGPT would be a 3 on this scale, 3.5 if you’re dumb enough to use it to power a semi-autonomous application.) (For more details on the levels of “AI”, see the detailed Pro piece the doctor wrote over on Spend Matters on how Artificial intelligence levels show AI is not created equal. Do you know what the vendor is selling?.)

However, thanks to ChatGPT/OpenAI and other offerings, every vendor all of a sudden feels that their solution has to have “AI” to compete, and is now claiming they have AI when, at best, they’ve implemented some third party “library” into their analytics module, which itself may or may not be AI, or, at worst, they just have classical rule-based automation and statistical-based predictive analytics (i.e. trend analysis) but have called it “AI” because, just like a classic decision-tree expert system from three decades ago, it can make a “recommendation”. Woo hoo.

Not that this is nothing new, three years ago a study by London Venture Capital Firm MMC found that 40% of European startups that are classified as “AI” don’t actually use AI in a way that is “material” to their business. MMC studied 2,830 “AI” startups across 13 EU countries, and in 40% of cases, [they] could find no mention of evidence of AI. (See the great summary in The Verge.) And even that statistic is a bit misleading, because I’m willing to bet that the “evidence” they did find was technology that didn’t necessarily mandate “AI” and could be implemented with “classical” techniques because, as a longtime blogger, analyst, due diligence professional and, most importantly, a PhD in theoretical computer science (read: advanced applied mathematics), I have found that most claims of “AI” weren’t really AI — in most cases they were just using a combination of automation and/or configurable rules and/or advanced statistics and/or machine learning and just had some of the foundations, but no real “AI”.

In our space, real “AI”, and by that I mean strong Level 2 / weak Level 3 (which is the best you can get) is quite rate and specific use cases are few and far between, and most AI is simply semi-unsupervised machine learning for transaction/categorical classification (spend analysis) or clause identification (contract analytics).

The problem is that, when no one really understands what “AI” is, and given that less than 1/10 Americans have the mathematical competency to even begin the university studies to try and garner an understanding [Level 4 on the PIAAC], it’s really easy form them to try and pull a fast one on you. This is especially true when the solution is able to automate certain tasks or recommend best practices in the majority of situations faster and more consistently than the average buyer (who, let’s face it, is under-educated — thanks to limited supply chain / operations management programs and almost no real Procurement training in Colleges and Universities, under experienced, and not an expert in modern technology), and the solution can be made to look “smart” (but, in reality, is dumber than a doorknob and definitely dumber than Maxwell Smart). But it’s not smart. Not at all.  And don’t be fooled.

The good news is the marketing manager using Applied Indirection to push a false AI solution at you probably doesn’t have a clue what they have anyway, and a few smart questions asked by someone who understands what AI is, and isn’t, can probably get pretty close to the truth pretty fast. For example:

1) “We have advanced AI data auto-class. It’s the most intelligent, and accurate, classification in the space.”

‘How does it work?’

“It uses a multi-level neural net that has been trained on tens of millions of records across over a hundred clients in the indirect space.”

‘Great, so basically it categorizes transactions based on similarity to other transactions in a slowly evolving manner, and I’m guessing for a new client in the indirect space, out of the box, you’re around 85% to 90% accuracy out of the box and you approach 95% with semi-supervised retraining over time — and that’s the upper bound and it will never be perfect.’

“Uhm, … well, … more or less … “

‘Got it!’ At this point you know it’s “AI” level for classification is augmented (as it learns and evolves over time), and barely, but it’s not “the best” mapping in the space as platforms that use AI to suggest rules (upon implementation and then for unmapped transactions) and do mapping and categorization based on the user selected and verified rules can produce 100% accurate mappings, always outperforming an “AI” solution that uses neural nets that are good (but not perfect).

‘Do you use AI anywhere else?’

“Uhm, what, why? It’s great where, and as, it is.

And now you know that there is no real AI in the analytics part of the platform, and there’s no reason to choose it over any other.

2) “We use AI for OTD prediction and risk in delivery prediction.”

‘Cool. What algorithm do you use?’

“Huh, what do you mean?”

‘How does the application compute the OTD and/or risk associated with the delivery.’

>Wait for the hand off to their “data scientist” …< “We use a blended least-squares method to produce a prediction function where, if there is enough data for the product, carrier, and lane, we’ll primarily use that data for the function, but if there’s not enough, we’ll use the most similar (using a mathematical distance function) product, carrier, and/or lane data … “

Is that AI, well, if there’s some sort of learning involved in the selection of “similar data” or recommendations as to parameter tuning IF parameters can be tuned, maybe, but this is just classical statistical trend analysis and not really any different than classical ARIMA based forecasting from the 70s, and did they have ANY AI then?!? (The answer is “NO”!)

3) “We use AI for our supplier recommendation process?’

‘Sounds promising … please explain!’

“We compute a relevance score taking into account a large number of factors including product base, geographic location, diversity, risk, etc.”

‘OK … how … ‘

>Cue the Eventual Hand Off to “Data Science” Team<

“Product Base is computed as a percentage of the category they can likely cover, geographic location as an average distance function, diversity as an estimate of diversity employment if there is no diversity ownership data (in which case it’s just 50%), the risk score from our risk model, etc. “

‘So, in other words, it’s just a formula … ‘

“A very sophisticated multi-level formula with conditionals and nesting that computes … “

‘Got it thanks!’ NO AI! Not even a hint there of as it’s just a functional risk score that could be built in ANY application with a formula builder.

This isn’t to say that a solution without AI isn’t right for you! (In fact, it probably is!) It’s all about solving your business problem, and many problems have been solved in our space just fine for the last decade or so with rules-based workflow and automation, optimization, and statistical modelling and trend projection. When guidance is needed, decision trees/matrices tied to expert curated best-practices (the modern equivalent of a classic “expert system”) often work better than one could imagine. In other words, it’s not AI, it’s not the hype, it’s what solves your problem, reliably and predictably time-after-time.

So don’t fall for the false hype and be the April fool.