In yesterday’s post, we noted that the latest Forrester Wave on eProcurement Solutions has been released and that Jason was right in his recent Spend Matters post when he said the Forrester ranking methodology, generally, does a better job than Gartner because it provides better transparency into the criteria that contribute to a ranking on each axis, but that, overall, the report wasn’t much better for a few, very significant, reasons.

First of all, limiting the evaluation to vendors with more than $15 Million in revenue is unduly restrictive. While it’s critical that the vendor have a steady income stream to maintain stability, a specialist vendor focused solely on best-of-breed e-Procurement solutions can be quite stable around the 5 Million mark. e-Procurement, like e-RFx and e-Auction, is a mature technology that’s not rocket science. As a result, a specialist provider with real talent in the R&D organization can easily maintain and continue to enhance such a solution on a regular basis with only 1 to 1.5 M in the R&D budget.^*1 And since a small provider without a lot of sales & management, operational, and executive overhead can devote 30% or more of revenues to R&D, it’s easy to see that 5M makes a very sustainable company.

I realize that lowering the threshold would have entailed a lot more work, and probably doubled the number of vendors, because then Forrester would probably have had to look at b-pack, Conexa, Coupa, Esize, Global eProcure, Intenda, iValua, Ketera, Proactis, PurchasingNet, Puridiom, Verian, and / or WaxDigital. But would that have been so bad? Especially when CapGemini and Hubwoo, which are just extensions of an underlying SAP platform, were included?

Secondly, while product strategy is important to consider in an evaluation (is this an end-of-life product or a product that is just hitting it’s stride with years of improvements planned), strategy does not deserve a 50% weighting. If a company is going to acquire a solution, it has to solve the problems the company has today, not solve those problems in two years time. Furthermore, while financial resources to pursue the strategy are also important, if the list is limited to companies with a sufficient revenue stream, it has no place in the weighting since all companies make the cut. And as for corporate strategy, that will be reflected in the product strategy. Plus, with respect to the “current offering criteria”, where’s the “integration” criteria? Remember, it is Sourcing and Procurement. Thus, while supplier connectivity and enablement is important, so is integration with the sourcing suite and the underlying ERP (that holds the organization’s data store, if there is one).

Third, procurement isn’t the problem, it’s compliance! It’s getting the big maverick spenders under control and forcing them to buy on contract (unless there is a real, management approved, need to go off contract) and forcing the suppliers to only deliver contracted merchandise and to bill for it at contracted rates. This is why most organizations only see roughly half of negotiated savings — mavericks buy off contract and they don’t catch the unapproved supplier substitutions and overbillings. Both require a good settlement function with advanced reconciliation and m-way matching capabilities. In the first case, invoices from contracted suppliers without POs have to be caught and denied (since the contracts will state no invoices without POs will be paid) and in the latter, matches have to be done against the PO and contract. However, “settlement” only gets a 1.5% weighting in the Forrester evaluation! In comparison, (future) product strategy gets a 30% weighting and goods purchasing, which an application has to have to be considered eProcurement, gets a 10%.

But, as I noted in my last point, at least this report had some good points. For better or worse, it defined inclusion and evaluation criteria and stuck with them. No vendors slipping in or out on analyst exceptions or technicalities. It displayed an understanding of what maverick spend is and why e-Procurement is needed to counter it, even if it didn’t capture and weight the appropriate functionalities. It noted that the right process must be easier than the rogue one and that approvals must be rapid when the right decision is made. It understands that while suites are getting better, there is still lots of edge left for best-of-breed to capitalize on. And it provides its evaluation spreadsheet to its clients who can alter the weightings to see which subset of the solutions are more appropriate to it based on the Forrester criteria. (You might still end up with a foot in the grave if you select one of these solutions, especially if you’re not a 1000 lb gorilla, but at least you can choose your own grave!)

Conclusion: Unlike many analyst reports, it’s definitely worth a read, but I wouldn’t base a decision on it unless you’re a 1000 lb Gorilla who is limited to buying from a 800 lb Gorilla by corporate mandate. An average mid-market company IS NOT likely to find the right solution for it in the evaluated vendor mix.

Finally, for those of you who have decided that you are going to limit your eProcurement selection to one of these gorillas, I’d watch for Jason’s forthcoming vendor analysis. SI will not be doing any further analysis on this report. Given that it missed the majority of solutions appropriate to the mid-market with its ultra-restrictive inclusion requirement and that a number of these vendors are in the very small set of vendors who won’t talk to SI, it’s not worth it.

^*1 There are some “micro” companies in the space that I follow that work magic on a yearly basis on an R&D budget that never tops 1M.

Ocean or Kiddie Pool?

As one of the flagship publications in the space, this is one that, for better or worse, a lot of people look forward to come decision making time. So, just like SI tackled the Gartner Quadrant last year, it’s going to tackle the latest Forrester Wave on eProcurement Solutions (Q1, 2011) to help you understand what’s good, what’s bad, and, in some cases, what’s downright ugly. Because, in the end, if you don’t know how to ride the wave, you might just end up digging your own grave.

First off, I agree with Jason (who commented that “Forrester’s eProcurement Wave Captures the State of the Market” on Spend Matters) that the Forrester ranking methodology, generally, does a better job than Gartner because it provides better transparency into the criteria that contribute to a ranking on each axis, that this report in particular does a solid high-level job of creating a credible segmentation for a sub-set of the vendors in the market, and that “there was little to broadly differentiate” among providers, at least on a feature/function level for providers that were included in the report. But better is not sufficient, high-level segmentations are pretty easy, and if you limit your report to the 800lb Gorillas, all of the solutions are going to pretty much look alike.

For example:

• if you have to get from New York to Los Angeles quickly, rail is better than car (because even though it makes lots of stops, the train runs 24 hours a day and you can’t drive 24 hours a day), but doesn’t match the efficiency of air and a direct flight
• there are lots of ways to credibly segment vendors — product focus vs service focus, e-Procurement focus vs ERP focus, generic solution vs vertical solutions — but such segmentation is meaningless to a buyer if it doesn’t segment according to the buyer’s particular needs
• if you limit your search to slivery mid-sized sedans, from a distance, there’s not much difference between a Toyota Camry, Ford Fusion, Nissan Altima, Honda Accord, or a Hyundia Sonata (and you’re likely to confuse them if you’re driving fast and just take a quick look)

In other words, while this was a little better than last year’s Tragic Quadrant from Gartner — where strict guidelines were set down but vendors allowed to slip in on exceptions or technicalities anyway, where some of the evaluation criteria didn’t make any sense at all, and where some non-standard definitions were used — it wasn’t much.

Basically, for just about every fundamental it correctly included, there was an accompanying flaw. And while most of the flaws weren’t that bad, the net result is that the overall report isn’t that useful unless you’re a 1000 lb Gorilla trying to figure out which 800 lb Gorilla you should buy from. And since there are only 1000 companies in the Fortune 1000 club, this means that the number of companies that will find this report useful are few and far between, and, as usual, the burgeoning middle-market, where most of the need is, goes unserved again, and the tsunami you might have been expecting is nothing more than a weak 6-foot wave that won’t do anything more than get you a little wet.

So what were the (major) flaws? That’s the subject of tomorrow’s post.