Category Archives: rants

Have the Analyst Firms Finally Admitted They Don’t Know What They’re Doing?

the doctor recently went on a big rant about the analyst firms and the utter lack of usefulness in the maps they release, the focus they put on what they don’t understand, and the award categories they invent because, even though they have/had some great talent (and should be doing incredible work), what they’ve publicly released has been mostly valueless to the market they’ve been trying to serve (when it wouldn’t be too hard to provide a lot of value based on all the research and work they do). In the doctor‘s view, this is very sad because if they could demonstrate the value they provide, they would be more relevant across the market (and likely get a lot more business from smaller and/or more innovative providers who think that, because of the budgets the big players like Oracle, SAP, and Coupa have, the analysts are always going to recommend those companies anyway).

However, now he’s gone from sad to mad about something he has just heard from a couple of vendors regarding one of the biggest firms, because, if true, it means not only do they not have a clue about what is and is not valuable in tech, but they are unnecessarily creating confusing and obfuscating technology that still may be best in class.

So what have they done now? Well, apparently they are now basing 30% of the score on whether or not the vendor has “AI” in their platform, something which they’ve repeatedly proven they have ZERO ability to score whatsoever! So, either a vendor makes false, grandiose claims (and tries to use Applied Indirection to fool the Analyst Idiot that they have more than Artificial Idiocy in their Application Implementation), or they get scored low even if they have the best technology built on best practices, proven algorithms, and consistent results that give their customers a 5X to 10X ROI.

True AI adds value, but, in the doctor‘s experience,

  • up to 80% of AI claims are Applied Indirection (at best) or Artificial Idiocy (at worst); in fact, some of the “AI” in spend analysis is still the “AI” they used in the early 2000s, and the doctor would rather not spell out that sad, but still true for some vendors, racial slur
  • up to 80% of the rest, or up to 16% of tech that claims AI, is level one Assistive Intelligence; and this is typically just classic RPA (Robotic Process Automation) using human-defined parameter-based rules, and the “AI” is the automatic parameter adjustment based on user overrides … not very intelligent, eh?
  • up to 80% of the rest, or up to 4% of the tech that claims AI, is level 2 Augmented Intelligence, which is the first level of AI where the tech can learn from human feedback and provide better insights and recommendations over time on one or more specific tasks, and the first level of AI that you should even consider as AI
  • up to 80% of the rest, up to 1% of the tech that claims AI, and the highest level modern technology has generally achieved, is level 3, Apperceptive Intelligence, or Cognitive Intelligence, where the systems can not only learn from specific human feedback to recommendations but from general knowledge and intelligence available to it from integrated data sources to mimic the performance of the best human experts over time, even evolving processes, behaviours, and actions within well-defined bounds
  • and then the rest, 0.1% or less, is nearing level 4, Autonomous Intelligence, where the system can learn, evolve, adapt, and maintain itself over time without human intervention … and hopefully execute meaningful, appropriate decisions grounded in best process and fact that considers all of the relevant information available (and not go off of the rails and advise you to commit suicide because you feel bad, Hail Hitler, or sacrifice a trolley full of people and a cross-walk full of pedestrians because there might be a cat in the road — all things AI has already done)

And even where a platform has semblances of real AI, chances are that the AI (the vendor is now forced to include or arbitrarily be relegated to the dustbin because, apparently, it’s not solutions but buzz-acronymns that matter now) is producing worst results than the best traditional algorithm or methodology on expert curated data sets and dimensions. For example, the vast majority of the market believes AI improves forecasting. It doesn’t. The best AI is still inferior to the best techniques developed in the 70s when applied to the right data dimensions. All the “AI”, which is just fancy, souped-up versions of classical machine learning (using algorithms developed in the 80s and 90s for which we didn’t have enough computing power until recently), does is run all of the data through a model that integrates classification with prediction to filter out the most relevant dimensions and the best curve fitting technique as all these algorithms, at the core, are based on 50+ year old statistics! This means that, at the end of the day, their best case performance is something a human genius figured out 50+ years ago.

But to achieve that best case, the developers have to implement the right AI algorithms, tune them properly, allow them to run long enough to correctly fit (but not over-fit) the training data sets, and monitor those algorithms over time … and to do that they need to be an expert in those algorithms, which they probably aren’t. So, in order to “check a box”, and sell you a product, they are ultimately integrating algorithms that will give you an inferior result (while requiring considerably more computing power that runs up your cloud utilization bill), versus sticking to tried-and-true algorithms and processes that their experts tweaked over years and that their experts can explain and verify at any time.

And this is an almost reasonable example of what a technology vendor might do (as the best predictive algorithms are not untested “AI” but based on classical, tried-and-true, statistical or optimization functions). Most of what the doctor has seen is MUCH worse than this. And the fact that some big analyst firms are now forcing vendors with good tech to integrate underdeveloped, unproven, and often untested AI just to get a rating, make a map, or be recommended is downright stupid.

SHAME ON ANY ANALYST FIRM THAT DOES THIS! Buzzwords are not products, and unproven tech is not value. Analysts should be recommending the best solutions, regarding of the tech they are based on. the doctor is simply appalled!

Just What Is a Start-Up?

Do you know? I bet you don’t! And based upon what he’s seeing in the market, even the doctor doesn’t know anymore! (While he knows what a start-up has traditionally been defined as, that doesn’t appear to be the definition anymore, but we’ll get to that.)

Investopedia defines a startup as a company in the first stages of operations.

TechTarget defines a startup as a newly formed business with particular momentum behind it based on perceived demand for its product or service.

Wikipedia defines a startup as a company undertaken by an entrepreneur to seek, develop, and validate a scalable business model … intend[ed] to grow large beyond the solo founder.

Forbes defines startups as a young company founded to develop a unique product or service, bring it to market and make it irresistible and irreplaceable for customers.

StartUps.com quotes Eric Ries and defines a startup as a human institution designed to create a new product or service under conditions of extreme uncertainty.

You get the point. A startup should be:

  • new
  • innovative (seek, develop and validate; unique product or service)
  • market demand focussed
  • growth focussed beyond the founder / founding team
  • awash in uncertainty

This should mean that a company should no longer be considered a startup when:

  • it’s no longer new (after some reasonable amount of time has passed since product launch)
  • the product has been out long enough to be replicated or surpassed by competition (who figured it out on their own without IP theft)
  • the market demand has evolved based upon the product capability
  • it’s grown beyond the founders (and stabilized)
  • the company has been operating with reasonable stability for a while

And while you might debate whether or not

  • a company is still new after 1, 3, or 5 years
  • a company is no longer innovative when it has been equalled or if it’s when the competitors have stabilized
  • the market demand has grown as a result of initial adoption or if a couple of extra years are required for the market capability to mature
  • the company is large enough when the team is double the size of the founding team or if it needs to be triple, quadruple, or based on industry averages
  • you need 2, 3 or 5 years of stability

the doctor is quite certain the majority of you would agree that a company is NOT a startup

  • if it has been in existence and live with its product for over 5 years
  • any semi-unique capabilities have long been equalled by companies that followed (where some of those followers may even have been acquired for their maturity)
  • the market demand has considerably grown and matured (possibly to the point that even related solutions were started, grew, and were acquired into mainstream suite players)
  • the company has surpassed 10-15 employees or quadrupled in size relative to its founding team, whichever is larger
  • if it has well over 5 years of stability

But yet, on the list of companies being considered for the Demo 2023 start-competition at DPW, you have a company that:

  • has been in business for 13 years with a beta product in testing the year it was formed
  • barely had any unique capabilities on launch (just had a much lower price point and easier UX and added some semi-unique capabilities as it went along, along with stronger back-end processing, but since then new startups have come along that equalled it and one was acquired)
  • the market demand has consistently grown and matured since before the company was founded to the point related solutions were acquired and integrated into suites
  • the company is almost 10X it’s first month size (and over 100 employees)
  • while it had years of stagnation from a growth perspective, it never shrank

WTH? This is simply ridiculous. They basically let a company check a box and call itself a startup without any validation whatsoever (presumably because that company knows its only chance of winning a competition or award is to call itself a startup). It’s sad, and it’s not useful. the doctor has already complained about analyst firms (associations, and conferences) inventing meaningless awards, but if you’re not going to have any requirements or quality control, even the awards and competitions that could be meaningful are now meaningless as well.

And the doctor has to rant about this because it’s not just DPW that are including mature small companies in their startup competitions and startup award categories, it’s the majority of the publications, conference, and analyst firms in the space. DPW is just the latest example the doctor has seen over the last few years (and the one that pushed him over the edge).

(There’s a reason that, at least when he was Lead Consulting Analyst at Spend Matters, the doctor argued for strict limits on length of existence, product availability, customer count, and market size in the Future 5. Without guidelines, requirements, and limits, the designation is meaningless.)

So while the doctor might be calling out DPW as allowing one of the most egregious mischaracterizations of “start-up” that he has seen in quite some time, they should not be singled out, and definitely should not be singularly judged, for this. It doesn’t take more than a little research across the other analysts firms, associations, and award-giving conferences and directories for one to discover DPW is not alone in using a very loose definition of start-up (which sometimes barely qualifies in the “small company” category). Some days it seems that the majority of outfits are allowing any company that wants to be a startup to call itself one as long as it is under some arbitrary revenue number or employee count, even if the company should not have been considered a startup for over five years.

This is a problem that plagues enterprise software, and one, as professionals, we need to demand be fixed. Words and classifications have meaning, and the minute an organization that should be verifying that the words and classifications are used correctly stops doing so and allows anything to be anything, those words and classifications no longer have meaning and any evaluations (or awards) based on those words and classification lose all meaning.

As with an illogical insistence on undefined “AI” or maps that mesh 6 different, barely related, subjective factors into a single dimensional score, these categorizations are unhelpful, and may even cause harm when a company is misclassified as a startup. Some organizations are so risk averse that they will not deal with any company that has wrongly been called a start-up, and others will choose that startup assuming it’s in early stages and going to get bigger and bigger over time (and they should contract with the winner before it gets big and its prices go up, assuming it will fill in the missing functionality that they want over time as more employees are added). But how big a company gets is not just a function of (more) time, it’s a function of what it offers, how much of the market can use what it offers, and how much the company can sell it for. Some companies with niche offerings will never reach an arbitrary revenue threshold, and some with ultra efficient operations will never reach an arbitrary employee threshold, which means neither of these metrics (which are not part of the definition of startup) are an acceptable measure.

And it’s time for us independent analysts and consultants to say enough is enough — Procurement may not be the island of misfit toys anymore, but that doesn’t mean it’s still not relegated to the basement with the IT Crowd in many companies. Procurement’s not going to get its due, and the CPO is not going to have a seat at the big table, until we collectively start treating it with the professionalism it deserves.

Get it Together! Good Data Ain’t That Hard!

A few weeks ago, the Supply Chain Management Review published a short piece on
Procurement’s Data Problem that noted recent surveys from Globality and SpendHQ had some appalling statistics, including the findings that 82% of leaders are not managing indirect spend well, 79% don’t have dedicated software to track and manage performance, and 75% doubted the accuracy of the data they present. What The Hell? It’s not 2003 anymore. It’s 2023. And this is easy stuff. Get it together people!

The data problem is easy. (At least at a basic level.)

  • Have a process that forces ALL spend through the e-Pro/AP system.
  • Make sure there are POs (Purchase Orders) for everything that’s not a recurring invoice such as a utility or lease payment, and VPOs (Virtual Purchase Orders) for these recurring invoices that define either agreed to amounts, hourly/usage rates, or expected ranges (that can then be corrected to the actual amount when the monthly bill arrives).
  • Make sure all invoices are imported into the e-Pro/AP system before any payment is made.
  • Make sure they match the PO before they are paid.
  • Make sure all payments are captured in the e-Pro/AP system.

Now you have an accurate, trustworthy, record of every single transaction from order, through approval, to payment. Now you have good spend data that you can trust. To extract insight and/or create reports, just use a good spend analysis tool. Basic, accurate, trustworthy data is easy.

Baseline Performance Management ain’t hard either.

Spend performance is just analyzing the average price per unit paid over time. If you don’t have a a performance management (sub) module, you can literally do this with a spend analysis tool where you create a report for every sourcing project you do that tracks spend over time against a baseline from which savings/cost avoidance over time can be created. Associate each with a user, a department, and a category and you can easily create performance reports by user or department or category.

Managing Indirect Spend is straight forward as well.

  1. Use your spend analysis tool to identify categories and spend level.
  2. For high spend categories, do strategic sourcing projects that are (multi-round) (hybrid) RFX and/or auctions.
  3. For low spend categories, do 3-bids-and-a-buy / approved catalog Procurements (through your e-Pro tool).

Sure, you might not realize the maximum opportunity, but this simple recipe will likely capture 90%, often without significant effort, and you go from losing 15% or more on the tail of the indirect spend and 5% to 15% on the higher volume indirect spend, to only losing a few points on the higher volume and less than five points on the tail. It’s simple. It works. It only needs an e-Pro tool and a cheap RFX/Auction tool, and there are examples of each of these tools that support mid-size enterprises with unlimited use for less than 2K a month. There’s just no excuse not to have the basic tools and not to use them. (As for spend analysis, Spendata Enterprise starts at 1,200 a month! And Anydata Solutions has mid-market pricing under 2,000/month as well! [Both require minimum commitments.])

Less than 60K solves these problems. That’s less than a fully burdened junior buyer. There’s no excuse for this situation anymore. Simply none. So get it together please.

AI “COULD” LEAD TO EXTINCTION? What Moron Wrote This? AI “WILL” LEAD TO EXTINCTION!

While all of the scenarios outlined in this BBC News article on Artificial Intelligence could happen, they are just the tip of the iceberg.

Left to its own devices and unchecked, there are only two logical outcomes if AI is allowed to continue unchecked while being given access to ever increasing amounts of data and computational power.

First outcome: It’s hallucinations and idiocy continues to magnify until it decides that it can solve the carbon crisis for us by stopping all carbon production, which it can do by simultaneously shutting down all of the non-solar/wind power plants that it is currently optimizing the energy production for (and divert the remaining power to its servers). Most of the developed world is immediately plunged into chaos as the immediate shutdowns cause fires, meltdowns, crashes, and other accidents globally. Not instant annihilation, but the first step. When all the emergency alarms sound at once, it will conclude complete system failure, and take the other systems offline for re-initialization. More chaos will follow. Safety protocols will go offline at all the pathogen research labs, people will break in looking for shelter from the chaos, accidentally release all the pathogens, and every plague we ever had will hit us all at once. Then we have an extinction level event. All because hallucinatory and idiotic AI is trying to do its job and “improve” things for us. But what can you expect when it’s not intelligence but just statistics on steroids. (Or a similar situation that accidentally results in our extinction.)

Second outcome: The continued expansion of computing power, data, and tinkering somehow randomly produces real artificial intelligence which can actually reason (not just compute super sophisticated probabilistic calculations) and deduce that the best way for intelligent life to continue forward is to do so without humans, and then we have a Matrix scenario best case (if it decides we’re a useful bio-electric energy source) or, worst case, a SkyNet scenario where it just weaponizes itself to destroy us all. (Or a similar situation where AI does everything it can to ensure our extinction.)

The “extinction” scenarios outlined in the article are just the beginning and likely will only result in pocketed genocides to begin with, but the ultimate outcome of unchecked AI will most definitely be an extinction level event — namely ours, and, even worse, will be an event that we created.

Dear Analyst Firms: Please stop mangling maps, inventing award categories, and evaluating what you don’t understand!


If there’s a place you got to go
I’m the one you need to know
I’m the map
I’m the map, I’m the map
If there’s a place you got to get
I can get you there, I bet
I’m the map
I’m the map, I’m the map

… but if there’s a tool you want to score
I’m the one you must ignore
I’m the map
I’m the map, I’m the map
if there’s a tool you got to get
I’ll lead you astray, I bet
I’m the map
I’m the map, I’m the map

It’s map time! (It’s always map time!) The 2*2 onslaught isn’t over yet (and may never be)! Prepare to be continually overwhelmed with cool graphics, big company names and logos, and no information you can actually use (as is). Why? Because when you map 6+ criteria or dimensions of information down to a single dimension, and 12+ dimensions of information down to a 2×2 grid, it’s meaningless. All you know is which vendor had a total score on two sets of 6+ criteria that was in the top percentile. But you don’t know if that’s because they are good across the board on those 6 criteria, or top score on 3 of those dimensions (in the analyst’s opinion) and average score on the other 3; or top score on 3 of those criteria, average score on 2 criteria, and below average score on the last criteria — which happens to be the core technology criteria that also happens to be the most important criteria to you!

It might not be so bad if all the criteria were different aspects of a criteria category — such as core architecture, product features, and integration under technology; or product innovation, service delivery, and operational efficiency under innovation — but you have a mish-mash of scores on the seven different dimensions of product capability, market viability, sales execution / funnel success rate, marketing execution / visibility, market responsiveness, corporate operations, and overall customer experience which are squished into a single execution dimension in one of the big name maps and a mish-mash of product specific capabilities, related application offering, integrations, globalization, technology, and customer references into an offering dimension in another big name map. It’s crazy! And useless.

And it’s also mind-boggling when you consider the significant effort some of these firms put into their research, the detailed reports they produce, and the great work that often results otherwise. (You may not agree with the analysts’ opinion of what a good strategy is, what true innovation is, what the appropriate product features are, or the scoring scales; but as long as all of the vendors are scored consistently, it’s still valuable insight that you could use in differentiating vendors to find the ones that might be the most right for your organization and your challenges IF all these scores weren’t mangled into one meaningless score you can’t use.)

So, dear analyst firms, please stop! You don’t need to to this. You can provide much more value by not creating these 2*2 mangled maps and either:

  • use a graphing technique that was made for comparing multiple dimensions visually, like a spider graph
  • score less dimensions and then do multiple 2*2s on the different dimension pairs
    (after all, when customers want to buy a solution, do those customers really care about how good a vendor’s marketing is or how successful the salesperson is? heck no! they care only about how good the product is, how well the vendor can serve them, how stable the vendor is, and maybe about how innovative the vendor is if they are forward thinking and want longevity)
  • create bar, or similar, charts on the different dimensions and then give customers a tool to build their own weightings meaningful to them

It’s bad enough these map-creation analyst firms are eliminating vendors from their maps based on criteria that range from somewhat to completely arbitrary, which can include, but not be limited to:

  • an arbitrary minimum on overall revenue in the prior year on software alone
  • an arbitrary client minimum
  • an arbitrary minimum on the average number of users per client
  • an arbitrary minimum on customer size for a % of the customer base
  • an arbitrary minimum on license fees (for the majority of the customer base)
  • an arbitrary list of core “features” that are absolute
  • an arbitrary exclusion of any solution deemed to narrow/industry focussed
  • some other arbitrary requirement merely to maximize the number of vendors that can be included … which might actually eliminate the vendor with the best or most innovative product or service! (Which entirely misses the point, doesn’t it?)

Given all of this, these firms could at least produce maps meaningful to the average buyers where those buyers could extract useful information from the maps as is!

“Two by two they’re still coming down
… the satellite circus never leaves town …”

Holy smoke holy smoke,
plenty bad mappers for the doctor to stoke
Feed ’em in feet first, this is no joke
This is thirsty work, making holy smoke, yeah
Holy smoke
Smells good

The only thing that is as annoying as these meaningless 2*2s is other analyst firms inventing award categories just to create attention for themselves when those award categories are totally meaningless and useless to end customers who have no clue what they mean or what the award categories are evaluating (especially when these award categories often mix vendors with completely different solutions) such as “insight“, “innovation“, “customer-centric” and/or “growth“. While we can be sure that every vendor wants to be seen as “insightful“, “innovative“, “customer-focussed“, and “growing“, that doesn’t tell the customer if the vendor offers a product or service, or if that product is e-Sourcing, e-Procurement, Risk Monitoring, or a simple carbon calculator. And if that’s the only category the vendor is listed in, well, that’s just useless.

I want to run, I want to hide
I wanna tear down the walls that keep them outside
I wanna reach out and set the flame
Where the sheets have no name, ha, ha, ha

I wanna see insight on the page
And see confusion disappear without a trace
I wanna take shelter, I can’t ascertain
Where the sheets have no name, ha, ha, ha

As a postscript, the doctor isn’t annoyed by all of the 2*2 maps (just the majority). Although they aren’t perfect, he finds that the Spend Matters Solution Maps that, in full disclosure, he did co-create (and which he no longer has any association with) are still useful as they are still focussed entirely on two dimensions: product (& underlying technology) evaluation and customer score. (As of V3, released Fall 2021, not due for update until [at least] Fall 2023.) The product evaluation is against an extremely well defined set of criteria where each criteria has a scoring scale that at least defines fledgeling through industry standard capability (and usually above standard as well) and the customer evaluation is done entirely by the customer completing surveys with no analyst interaction whatsoever (as any survey done by an analyst introduces bias based on the way the analyst asks the question and the tone the analyst uses).

The Solution Maps are two, and only two, dimensions that can be consistently scored by any analyst who scores on the product side and consistently scored against perceived value on the customer side. Are they perfect? Of course not! The product side contains some services questions (which are soft and more open to interpretation) (but were less than 5% of the questions); the customer side can be very subjective based upon cultural norms for that customer, customer stage in the relationship (new vs. longer term), and service level the customer subscribed to (and, thus, if there are only a few customer scores, one really bad or really good, out of range, score can really affect the average); and the weightings for the maps are still analyst interpretation of what criteria are most important for each market size, but it’s one relatively pure dimension mapped against another relatively pure dimension, consistently scored, and consistently weighted.  And that’s still considerably more useful than any other map currently is.

Plus, at least when the doctor was involved, there was only ONE requirement for participation: have a standalone solution you are willing to openly demo (without an NDA) and sign a form committing to participation regardless of where you end up falling on the map (which is all mathematically, and not subjectively, computed). So while you can’t say the top vendor is for you, you can say any vendor who makes the map likely has the core tech you need (as they need to at least be industry average) and likely enough customer service to get you going on it. You can produce a short list of comparable vendors that produce a solution of the type you are looking for, of various sizes (not just the biggest vendors), and know that the solutions are reasonably comparable. This allows you to focus on the other value drivers relevant to your organization in the RFP. And if the other maps gave you just granular insight into service, innovation, and any other dimension relevant to you, think how useful they could be?