Category Archives: Analyst

Have the Analyst Firms Finally Admitted They Don’t Know What They’re Doing?

the doctor recently went on a big rant about the analyst firms and the utter lack of usefulness in the maps they release, the focus they put on what they don’t understand, and the award categories they invent because, even though they have/had some great talent (and should be doing incredible work), what they’ve publicly released has been mostly valueless to the market they’ve been trying to serve (when it wouldn’t be too hard to provide a lot of value based on all the research and work they do). In the doctor‘s view, this is very sad because if they could demonstrate the value they provide, they would be more relevant across the market (and likely get a lot more business from smaller and/or more innovative providers who think that, because of the budgets the big players like Oracle, SAP, and Coupa have, the analysts are always going to recommend those companies anyway).

However, now he’s gone from sad to mad about something he has just heard from a couple of vendors regarding one of the biggest firms, because, if true, it means not only do they not have a clue about what is and is not valuable in tech, but they are unnecessarily creating confusing and obfuscating technology that still may be best in class.

So what have they done now? Well, apparently they are now basing 30% of the score on whether or not the vendor has “AI” in their platform, something which they’ve repeatedly proven they have ZERO ability to score whatsoever! So, either a vendor makes false, grandiose claims (and tries to use Applied Indirection to fool the Analyst Idiot that they have more than Artificial Idiocy in their Application Implementation), or they get scored low even if they have the best technology built on best practices, proven algorithms, and consistent results that give their customers a 5X to 10X ROI.

True AI adds value, but, in the doctor‘s experience,

  • up to 80% of AI claims are Applied Indirection (at best) or Artificial Idiocy (at worst); in fact, some of the “AI” in spend analysis is still the “AI” they used in the early 2000s, and the doctor would rather not spell out that sad, but still true for some vendors, racial slur
  • up to 80% of the rest, or up to 16% of tech that claims AI, is level one Assistive Intelligence; and this is typically just classic RPA (Robotic Process Automation) using human-defined parameter-based rules, and the “AI” is the automatic parameter adjustment based on user overrides … not very intelligent, eh?
  • up to 80% of the rest, or up to 4% of the tech that claims AI, is level 2 Augmented Intelligence, which is the first level of AI where the tech can learn from human feedback and provide better insights and recommendations over time on one or more specific tasks, and the first level of AI that you should even consider as AI
  • up to 80% of the rest, up to 1% of the tech that claims AI, and the highest level modern technology has generally achieved, is level 3, Apperceptive Intelligence, or Cognitive Intelligence, where the systems can not only learn from specific human feedback to recommendations but from general knowledge and intelligence available to it from integrated data sources to mimic the performance of the best human experts over time, even evolving processes, behaviours, and actions within well-defined bounds
  • and then the rest, 0.1% or less, is nearing level 4, Autonomous Intelligence, where the system can learn, evolve, adapt, and maintain itself over time without human intervention … and hopefully execute meaningful, appropriate decisions grounded in best process and fact that considers all of the relevant information available (and not go off of the rails and advise you to commit suicide because you feel bad, Hail Hitler, or sacrifice a trolley full of people and a cross-walk full of pedestrians because there might be a cat in the road — all things AI has already done)

And even where a platform has semblances of real AI, chances are that the AI (the vendor is now forced to include or arbitrarily be relegated to the dustbin because, apparently, it’s not solutions but buzz-acronymns that matter now) is producing worst results than the best traditional algorithm or methodology on expert curated data sets and dimensions. For example, the vast majority of the market believes AI improves forecasting. It doesn’t. The best AI is still inferior to the best techniques developed in the 70s when applied to the right data dimensions. All the “AI”, which is just fancy, souped-up versions of classical machine learning (using algorithms developed in the 80s and 90s for which we didn’t have enough computing power until recently), does is run all of the data through a model that integrates classification with prediction to filter out the most relevant dimensions and the best curve fitting technique as all these algorithms, at the core, are based on 50+ year old statistics! This means that, at the end of the day, their best case performance is something a human genius figured out 50+ years ago.

But to achieve that best case, the developers have to implement the right AI algorithms, tune them properly, allow them to run long enough to correctly fit (but not over-fit) the training data sets, and monitor those algorithms over time … and to do that they need to be an expert in those algorithms, which they probably aren’t. So, in order to “check a box”, and sell you a product, they are ultimately integrating algorithms that will give you an inferior result (while requiring considerably more computing power that runs up your cloud utilization bill), versus sticking to tried-and-true algorithms and processes that their experts tweaked over years and that their experts can explain and verify at any time.

And this is an almost reasonable example of what a technology vendor might do (as the best predictive algorithms are not untested “AI” but based on classical, tried-and-true, statistical or optimization functions). Most of what the doctor has seen is MUCH worse than this. And the fact that some big analyst firms are now forcing vendors with good tech to integrate underdeveloped, unproven, and often untested AI just to get a rating, make a map, or be recommended is downright stupid.

SHAME ON ANY ANALYST FIRM THAT DOES THIS! Buzzwords are not products, and unproven tech is not value. Analysts should be recommending the best solutions, regarding of the tech they are based on. the doctor is simply appalled!

A 60 Minute Call is NOT Due Diligence!

It used to be the doctor would only get a request once every month or so for a “call with a client looking for some insight into the space from an expert“, but now it’s the case he’s getting these every week, often multiple times a week, from yet another firm that “specializes in connecting clients looking for insight with experts” or some other such meaningless gobblydygook from a knock-off Dilbert Mission Statement Generator.

Maybe it wouldn’t be so bad in the grand scheme of events except,

  1. You can’t learn anything meaningful in 60 minutes. (We’re talking enterprise software solutions, not the results of an investigative whodunnit.)
  2. These requests are now coming from kids so young the doctor is wondering if they are still high school (despite the fancy LinkedIn titles their firms give them) … and not to be ageist, but there’s no way these kids have any deep understanding at all of any industry domain or what makes an expert (and how to judge if that expert has the right education and experience).
  3. It seems companies are using a handful of these calls as “due diligence” on a space or a company.

And a 60-minute call is NOT due diligence. the doctor does product and technical due diligence, and even a high-level due diligence on a company (which is just looking for potential red-flags and yellow-flags that will have to be watched) takes weeks of man power (as the team he worked with did market, strategy, product, and technology, and even though the doctor can do an entire product and technical diligence in S2P on his own — no team of 6 to 10 needed — it’s at least a week of effort on a single module to get enough certainty that there are no red flags and the important yellow flags have been identified). This is because a due diligence involves process reviews, document reviews, code reviews, focussed interviews, etc. etc. etc. and comparisons to standards, best practices, and market norms.

Given this, just what are you going to learn from a few call with external “experts” who don’t have any access to documents, processes and practices, and the internal stakeholders who make the decisions? Opinions. Maybe. But most likely, absolutely nothing!

In other words, if you need deep insight, find an analyst, diligence, or strategy firm that knows the space and, if you are interested in a company in particular, find an analyst, diligence, or strategy firm that that knows that company AND that company’s peers. And go with them. Don’t pay for the privilege of paying for the privilege to talk to someone who won’t end up being that useful to you. Especially if you need to be able to back up a(n investment [related]) decision that involves the company and prove you did your homework and the stars were aligned as well as they could have been when the decision was made (since no one can predict the future, just play the odds).

In other words, these firms, which the doctor will have nothing to do with, need to go away. A consultant who has the expertise to find the right analyst / diligence / consultancy for you and introduces you to the right individual in that firm deserves a finder’s fee, but doesn’t deserve a fee for the privilege of hooking you up with a random yahoo who can’t help you at all. (And even if that individual is an expert in their area, if it’s not the area you need, and they know next to nothing of relevance in the area and company relevant to you, they’re a yahoo from your perspective.)

And as you probably figured out by now, if you reach out to the doctor and he’s not the right expert for you, he’ll pass you on to someone he believes can (which could be one of the 40 experts he explicitly mentioned, and linked to, in yesterday’s post). It’s not about “sign the contract at all costs and hope to figure it out later“, it’s about helping your prospect because, even if they don’t become a client today, when their need is appropriate, they will become a client tomorrow.

Seeking an Analyst? Who does the doctor recommend?

In our last two posts, we asked how relevant is the analyst firm and then answered that it’s not the analyst firm that’s relevant, but the senior analysts in its rank that are relevant. (And if the firm doesn’t have any in your Source-to-Pay/Supply Chain (related) area, it doesn’t matter how many employees it has, how many countries it is in, how many Billions or Trillions its customers spend, etc. because it won’t be able to help you get your message right, hit your market, or enhance your strategy.)

So, in their honour, here are forty analysts (even if they didn’t work for analyst firms and mainly did behind-the-scenes analyst work as a consultant/advisor) that the doctor has trusted and often referred inquiries to over the years, past and present, and (some of) their (former) areas of specialty:

Present (note that many [14/25] are independent and NOT with a big firm):

Andrew Bartels Back-Office Tech-Driven Business Transformation Forrester
Andrew Karpie CWM/VMS/HCS Independent
Bertrand Maltaverne Source-to-Contract Spend Matters
Bob Derocher SC/Procurement Process Transformation Digitization Independent
Bob Ferrari Supply Chain, Manufacturing, Digitization Strategy Independent
Brian Sommer ERP, HR, & Finance (Transformation) Independent
Chris Sawchuk Metric-Based Procurement Modernization Advisory Hackett Group
Doug Smock Supply Chain Evolution Independent
Garry Mansell Entrepreneurship and Business Growth in S2P and SC / Global Supply Chain Design and Management Independent
Jason Busch Broader Source-to-Pay Market Strategy Spend Matters
Katie Evans AI Ethics Independent
Kelli Coviello Business Growth, Diversity, Work/Life Balance Independent
Lora Cecere Supply Chain Supply Chain Insights
Mickey North Rizza Sourcing, Procurement, Commerce, SRM, Risk IDC
Navroop Sahdev Digital Economy Digital Economist
Noha Tohamy Logistics, Supply Chain Digitization, Analytics Gartner
Patrick Connaughton (Ecosystem) Enterprise Applications Gartner
Pete Loughlin Purchase to Pay / Procurement / Coupa & Ariba Independent
Peter Smith Best Practices, Sustainability, Procurement with Purpose Independent
Phil Fersht Emerging Technologies, Automation, Outsourcing, Global Business, and Horses HFS Research
Pierre Mitchell Procurement and Services Benchmarking & Transformation Spend Matters
Robert Rudzki Strategic Advisory and Procurement Transformation Independent
Sigi Osagie Business Growth through Personal and Team-Based Growth Independent
Vinnie Mirchandani Enterprise Applications and Outsourcing Independent
Voitek Szewczyk Strategic Sourcing, Procurement Transformation, Eastern Europe Independent
Xavier Olivera Procure-to-Pay/LATAM Market Spend Matters

Past ([semi-]retired, out of the analyst world, and/or working for a vendor; 4 independent):

Charles Dominick Procurement and Procurement Training Independent
Debbie Wilson ERP & Finance Independent
Dick Locke Operations, Strategic Sourcing, and International Trade Independent
Doug Hudgeon Back Office Integration & Modernization / Australasia Market Managed Functions
Duncan Jones Procurement Independent
Gerraint John Sourcing, Procurement, SRM, and Risk Interos
Jon W. Hansen Procurement Independent
Magnus Bergfors Strategic Sourcing, Strategic Procurement, SRM Keelvar
Mark Perera Procurement and SRM Vizibl
Nick Heinzmann Procurement, CLM, Sustainability, Fraud Risk, and Startups Zip
Sudy Bhardadwaj Direct Supply Chain, Source-to-Supply, Entrepreneurship SAP
Tim Minahan Procurement & Supply Chain, Business Performance Benchmarks, Best Practices Citrix
Tony Poshek Strategic Sourcing Simfoni
Vance Checketts Supply Chain, Operations Built for Teams
Viktoria Sadlovska Anshu Supply Chain, Trade Finance, Analytics RepTrak Company
Vishal Patel CLM and P2P Ivalua

It’s not the Analyst Firm. It’s the Analyst!

In our post on Friday that asked How relevant is the Analyst Firm?, we noted that, these days, I’m hearing far too often from new companies or smaller companies that weren’t acquired in the M&A mania that their [marketing] strategy is to “get on Analyst Firm Map XYZ” or “get in front of the big analyst firms as fast as possible and, hopefully get written up“.

And this scares me because,

1) as pointed out in our last post, they think “the firm” is the answer, when, in fact, it’s not the firm but the analyst because “the firm” will only get it right IF the analyst gets it right (and, if you get a junior analyst, you may find that they over-promote a competitor with great marketing and misleading AI claims but limited capability over a unique solution you offer that, due to the subtlety of the power at the solution core, the analyst isn’t able to grasp what he’s unable to see)

2) for an analyst to get it right, that analyst needs, at least, a dozen skill sets that, combined, require
a) an education that sometimes goes beyond an average PhD and includes
i) the equivalent of a bachelor’s in mathematics
ii) the equivalent of a bachelor’s in computer science or engineering
iii) the equivalent of a Master’s in Procurement or Supply Chain or
Advanced Operations Management (a Bachelor’s in business ain’t enough)
b) at least a decade of experience in the space to understand the breadth of technology, industries, and current capabilities
c) exceptional analytical skills (and questioning skills)
d) great writing skills (in a day where it seems no one can write anything without AI, but AI is only as good as the content sources fed into it, and those raw sources have to come from … that’s right … a human!!!)

3) the number of senior analysts we’ve had with the right education and experience has always been few and far between (with even the biggest firms never having more senior analysts in our space than you can count on the fingers of one hand at any one time), but with the departures / retirements of the majority of the best analysts in our space from Gartner, Forrester, and Hackett*2 over the last few years, and almost half of the senior analysts from Spend Matters …

that’s not leaving many senior analysts, or viable analyst firms, left, and, at least in the doctor‘s view, all of the firms except Spend Matters have been gutted*3 in our space at least once over the last few years, and, given the breadth and depth of requirements to be a good analyst, where’s the next generation of senior analysts going to come from?

[Unless another visionary in our space with a strong tech background, a couple of decades of domain experience, and great analytical and writing skills is willing to jump the fence to the analyst side, we’re not going to be getting many new senior analysts that we can rely on. They’re not at other analyst firms (outside our space), they’re not at consultancies, and the reality is that there’s only a handful of visionaries left that didn’t make their millions and retire already or who are still thrilled by the space and want to stay in it as long as possible.]

It wouldn’t be unrealistic to say that Bertrand Maltaverne could be the last great analyst in our space.

In other words, if you want to be sure you’re getting the right coverage, review, or feedback, you need to STOP assuming the analyst firm is the answer and start looking at the analyst inside, or outside, that firm (and further remember that many of the senior analysts who are still in the game are on our own for various reasons), find the right analyst for you, and make sure you get in front of that analyst (or don’t bother with the firm at all). And you need to further realize that it’s not possible for every analyst to be an expert in every technology in the Source to Pay space. You need the right review and guidance from the right senior analyst, or the end result is that it will be worse than no review and guidance at all.

 

*2 but, in fairness, we will point out that Gartner and Forrester have been aggressively working on replacing them, although this has often required poaching from other peer firms, so the number of senior analysts hasn’t increased by much 🙁

*3 one has to remember that, in addition to vendor poaching, there was M&A in the analyst space too, and this wasn’t always for the better! Especially when the acquirer worked to a beat or a model that was different then the acquired firm that itself was only successful because it was different and had the right people who worked well under their own unique beat or model.

In Source-to-Pay, How relevant is the Analyst Firm?

As a result of the M&A mania in the late teens (during the era of [mega] suite consolidation) and very early twenties (during the pandemic when all of the PE firms suddenly realized that e-Sourcing, e-Procurement and, most importantly, e-Payment solutions were critical [when no one could go into the office]), a lot of known smaller, and even mid-size, players were swallowed up, leaving a vacuum at the lower end of the market.

As a result, two things have happened:

* a slew of new players (run by leaders new to the market) have entered the market; and while most have very limited solution breadth or depth, their use of modern technology is plugging a hole and offering value out of the gate (especially to smaller companies with nothing)

* a lack of talent (which has also been swallowed up into larger companies) at the remaining offerings has resulted in many of the leaders in these companies coming from other areas of enterprise software

… and neither of these sets of players have a deep understanding of our market or the analyst firms in it and too often I’m hearing that part of the strategy is “get on the Gartner Map“, “get on the Forrester Map“, or “get on the Spend Matters Map“. And while the last map*0 is the map you definitely want to be on at some point (as it actually focusses on technology vs. a mix of soft vs. hard factors that make it hard to judge how technically relevant the solution on its own is for you), “getting on the map” isn’t a strategy.

As a corollary, I’m also hearing too often that a big part of the marketing strategy is to “get in front of the big analyst firms as fast as possible and, hopefully get written up“, and if there are analyst relations, all their time is focussed on these big firms. And that worries me. A lot!

Why? Because they think “the firm” is the answer, when, in fact, it’s not the firm but the analyst because “the firm” will only get it right IF the analyst gets it right. And at many of these firms, I’m more worried by the year if the analyst will get it at all. Why?

If we go back to Saturday’s post on AI: Applied Indirection, Artificial Idiocy, & Automated Incompetence, we have the dual problem that most of the solutions out there are claiming capabilities they don’t have and even most people in technology can’t judge whether or not the claims are real or fake, and this goes for analysts too. Especially new, junior, analysts without the right tech background, domain understanding, education*1 and experience in our space.

The reality is that we’ve went from the point where, in the beginning, to be a good analyst you needed to:

* understand the space
* understand the unique processes the technology has to support to serve the space
* understand the current breadth of offerings and capabilities across the vendor landscape

to where, to be a good analyst as technology progressed, you also needed to

* understand the different technology stacks and what they can, and cannot, offer
* understand the different technology options and what they can and cannot do (i.e. algorithms, workflows, etc.)
* understand the nuances of buyer needs across industries and niches (e.g. direct vs indirect, manufacturing vs. distribution, F&B vs CPG, etc.)

to today where, to be a good analyst, you also need to

* understand the different technologies that are used in ML/AI and what actually qualifies as ML/AI and what does not
* understand where advanced technologies, especially those based in ML/AI, are required, and where classic techniques will do just as well, or better
* understand the different levels of analytics, and whether a solution has real analytics, or just pre-packaged reporting
* understand how the different technologies on the market need to link together as we move from the world of suites to platforms

In other words, we’ve gone from the point where to be an analyst, in the beginning, you just needed:

* critical thinking skills
* a basic business understanding
* good writing skills

to where, as technology progressed, you also needed:

* a basic understanding of technology (2 years of computer science or equivalent STEM offering in engineering, physics, etc.) and scientific thinking
* a basic understanding of source-to-pay and related processes across industries and category uniqueness that may or may not dictate different needs
* a basic understanding of integration points to other enterprise systems
* a good domain understanding of the Sourcing/Procurement needs in modern multi-nationals

to where, looking at technology today, you also need:

* a deep understanding of math and analytics (and at least a Bachelor’s in a STEM area)
* a deep understanding of models and metrics and where, and how, all the different data sources integrate for risk, diversity, spend, and opportunity models
* a deep understanding of what’s needed for a modern data interchange, API integration, and procurement management platform
* a deep understanding of how procurement works with and supports supply chain, logistics, and finance and how the pieces support this
* a good bullsh!t detector and the ability to dive into claims that a company may want you to take without question and find out what really is there and what the claim really means
* at least a decade of experience on top of close to a decade of education (because if you’re not a genius, you probably need at least a Master’s or two Bachelor degrees to get all the background you need) to put it all together

But who has that anymore? And where are they?

To be continued … in Part II

 

*0 as of posting as those maps, V3, were designed as pure-tech [and the last iteration co-designed by the doctor]

*1 most programs, if they teach anything at all, teach classical operations management or logistics, neither of which is modern procurement or supply chain management, and definitely not advanced math or algorithms!