Category Archives: Spend Analysis

There’s Still No Spend Analysis Without the Slice ‘N’ Dice


SI originally ran this post 10, yes ten, years ago today, and nothing has changed. Regardless of how fancy that drill-down dashboard is, how many pre-canned reports come with the system, or how many sub-views you can create, if it’s still off of 1, that’s one, cube, it’s still limited in the value you will get. Moreover, this post is especially relevant because it reminds us of how BIQ changed the spend analysis game and that the individuals that spearheaded the company are game changers. This is particularly relevant because Eric Strovink, the founder and the person that has now changed the spend analysis game twice (first at Zeborg, now part of [IBM] Emptoris, before BIQ) is going to be launching a new analytics company this year, and chances are it won’t just be the same-old same-old rebranded Tableau or QlikView solution).

When I was in Boston, I was lucky enough to spend the better part of the day with Eric Strovink of BIQ, and have a few extended conversations with individuals at some of the local consulting firms that specialize in sourcing, and am now more than convinced that any tool that mandates a single cube, or makes it difficult to change the cube, is not a spend analysis tool, merely a spend data warehouse with built in canned reporting (and, if you’re really lucky, limited ad-hoc capabilities).

Not that there’s anything wrong with a centralized spend warehouse with a consistent view of your total spend, especially one that integrates multiple internal and external data sources and allows you to drill down and understand your spend at a detailed level. Of all the e-Sourcing software tools, it is the one most likely to make your CFO do backflips, especially if it has good reporting (and this is a big if – not all spend analysis tools on the market do), since it makes it really easy for the CFO to tell the CEO where the money is going and comply with all those pesky reporting requirements.

However, the value of such a tool is quite limited to you as a purchasing agent. Now, it’s true that the first time you’ll use it you’ll save big-time, especially if it’s the first time you have visibility into the majority of your spend, but the reality is that this is the only time you’ll see such significant savings. After you’ve identified all of the low hanging fruit identified by the single view provided to you by the system, analyzed each instance of over-spending, and taken corrective actions, you’ll find that you’ll be unable to identify additional savings and the system will simply function as a glorified data warehouse that you only use once a quarter to create those reports for your CFO and check that your teammates our buying off the negotiated contracts – something that you could do almost as well with your existing ERP system and a significantly cheaper Business Intelligence / OLAP tool like (SAP) Business Objects or (IBM) COGNOS and some grunt work.

Remember, I’m not saying that traditional spend analysis systems like those provided by e-Sourcing providers like (SAP Ariba) Procuri and (IBM) Emptoris are not without value – if you do not have a good, integrated, data warehouse that integrates your various accounting, purchasing, and inventory systems to provide you a single view of your spend or a good reporting system to produce all of the reports your CFO needs, then you’ll find these systems very valuable. However, it’s important that you understand that the primary value of these systems is in the total spend visibility they provide from a financial viewpoint, not the spend analysis capability you really require to identify potential overspending and cut-costs, because you’ll only be able to do this once – thanks to the single organizational view they are built on. (In other words, you’ll save big when you fist implement the system but future savings will be limited to your capability to quickly catch and stop maverick spend.) So, if you need a system to consolidate your spend data, produce the tedious reports required by all of the new financial reporting requirements, and give you some basic across-the-board spend visibility, or, more importantly, you need a spend data warehouse that integrates with the rest of your e-Sourcing suite, be sure to check these systems out – but understand what they are really worth to you before you sign the check.

In order to help you understand where these systems fail in true spend-analysis, why you need to be able to dynamically create multiple cubes on the fly which support dynamic dimensions, meta-aggregation, cross-dimensional roll-ups, and even federated data sets, I’m happy to inform you that Eric Strovink has agreed to co-author a series of posts outlining what real spend analysis is, how it differs from basic spend visibility, what it does for you, and why you need to get there.


Eric Strovink was actually kind enough to contribute two insightful series to Sourcing Innovation. Here are the links for your reference.

I: The Value Curve
II: The Psychology of Analysis
III: Common Sense Cleansing
IV: Defining “Analysis”
V: New Horizons I
VI: New Horizons II

I: It’s the Analysis, Stupid
II: Why Data Analysis is Avoided
III: Crosstabs Aren’t Analysis
IV: User-Defined Measures, Part I
V: User-Defined Measures, Part II

Spend360 – Applying Deep Machine Learning to Spend Analysis

Regular readers will know that, generally speaking, the doctor has not been impressed with the auto-classification and mapping offerings by any spend analysis vendor he’s ever blogged about as all have failed pitifully on tail spend, performed poor on any supplier or category the provider hasn’t processed extensively, and worked poor in new geographies and even poorer in foreign languages.

However, this year, he’s been impressed by two vendors with auto-classification. TAMR, which are trying to tame the data deluge, and now Spend360. While a new name on this side of the pond, it is not a new name across the pond, having opened its doors for business in 2011, after two plus years of intense development. Plus, it is gaining reputation pretty quickly since it’s foray to this side of the pond a couple of years ago and now has over 100 North American clients, which brings its total client base to over 400 global customers, which is impressive for any company in this space. (Even more impressive is the fact that, to date, it has processed over 1 Trillion of spend.)

While it’s still not perfect, and still can’t outmatch the best human expert with a multi-level priority mapping engine, it is decades ahead of its competition and has the ability to learn and evolve and, over time, approach 98%+ mapping accuracy, leaving little that has to be mapped, or corrected, by a human user (which is quite valuable when the user is not an expert in spend analysis but still wants to reap the benefits).

Not only can its deep machine learning identify tail spend suppliers, company specific categories, and even individual items coded in obscure ways, but it can learn over time and adapt to different data models, especially since it can use evolving knowledge bases. Whereas the majority of first generation classifiers used naive statistical classification that could not learn and had to map to a fixed (UNSPSC) model, Spend360’s uses deep machine learning (based on LSTM and encoder/decoder technology) that maps to custom data models using extensible knowledge bases (which can be created and maintained by the organization) that can encode organization and industry specific knowledge (and negate the need for custom mappings or override rules).

The fact that the knowledge base can be extended anytime a mis-classification occurs negates the need for manual mappings or override rules common in so many first generation spend analysis systems is a very powerful concept. It means that every erroneous mapping need only happen once and will never need to be manually corrected again. Plus, the fact that the data model can be extended as analytic needs evolve means that the platform can continue to deliver value year over year over year, unlike most first generation platforms that only delivered top N reports and failed to deliver value after the first twelve to eighteen months.

But this isn’t all Spend360 has to offer. In addition to a powerful classification ability, which can be trained to actually work, it also has a very powerful front end that allows the user to drill through the cube using custom filters in real time, compared to first generation systems that had fixed OLAP with limited filter capability. Reports can be cross-linked and all linked reports auto-update as one is drilled into. And data can be uploaded and incorporated into the cube in real-time if additional data is required.

And, to top it off, based on the 1 Trillion in spend they have classified over the years, Spend360 also has deep spend benchmarks across all of the major verticals and categories, which is often mapped down to UNSPSC level 4. This allows an organization to quickly understand how its spend on a category compares to the average in its vertical. Simply augmenting this data with pricing trend data can give an organization quick insight into where some significant cost normalization opportunities may lie.

In short, Spend360 is a provider the doctor expects you to be seeing a lot more of in the years to come, and recommends that you check out the upcoming deep dive, co-written with the prophet, over on Spend Matters Pro [membership required] if you are able. This is one best-of-breed provider you want to know.

True Savings Can Only Be Identified through Multi-Factor Optimization

A recent guest post from a vendor-employed guest contributor over on Spend Matters said to “Calculate Your True Savings Using Predictive Analytics”. While the doctor agrees predictive analytics can often give you a good data point as to projected savings, the reality is that it’s not always as accurate as you would like to believe and typically does not capture your best savings opportunities.

Why? Before we discuss the guest post, which did have some good points, we have to note that most predictive analytics algorithms work on trending and statistics on historical or market data, and while this can be highly accurate (95%+) the majority of the time (95%+), because market data is only historical and typically does not include data points on new (not yet introduced or announced innovations), detailed cost breakdowns on consumer / market prices, or operational insights into hidden inefficiencies whose correction can do more than shaving a few points off the top.

Going back to the post, the author states that if you use a Savings Regression Analysis (SRA) model based on multivariate regression of past-realized savings for a given subcategory to compute the savings potential under current market conditions, the target computed will be realistic, achievable, and likely mirror what you will do (despite the savings targets you set).

And this statistically based model will work if it is the same buyer (group) employing the same strategy on the same market base under similar conditions, but what could happen if a new buyer comes in that totally redefines the demand and the market strategy, or the market conditions have suddenly changed from supply shortage to supply surplus, or new production technologies could revolutionize production and trim overhead 20%? In this situation, this type of model will be significantly off.

Now, anything you can do to better predict savings is a positive, because, as the author points out, this allows for

  • better cash flow management (as you will better know your costs)
  • time to market optimization (as you will know the best time to source if you have leeway)
  • goal setting (as you won’t be trying to achieve the impossible)
  • performance management (as you can track against a realistic goal)

But while predictive analytics give a good data point, the best data point is when you use your market intelligence to build good should cost models, use optimization to minimize transportation and incidental storage and sales (and even taxation) costs (when sourcing globally), and use six sigma analysis to see if there is any opportunity to take cost out of a supplier’s overhead production cost. Going into this level of detail may indicate that while the product cost is likely to increase 1% this year (and explains why the predictive software says only 2% savings should be expected after heavy negotiations), an extensive analysis could show that a transportation network redesign could shave 3% and lean process improvements at your supplier could shave 2%, meaning that a cost reduction of up to 7% could be achieved with the right footwork (which is something the predictive model will never tell you). So use the predictive algorithms to establish a baseline, but never, ever stop there.

TAMR – Trying to Tame the Data Deluge!

TAMR may be a relatively new entrant in the stand-alone best-of-breed spend analysis space, having been incorporated back in 2013, but — and this is largely due to the pedigree and experience of its founders and senior team — it’s AI-backed probabilistic machine learning engine is on par with any player out there and it’s spend analytics success at some of the Fortune 500 players that have adopted it is on par with companies that have been doing spend analysis for over a decade.

And while, at first glance, TAMR appears to play in a large spend analytics space, when you zero in, you find that the vast majority of players offering spend analytics are (sourcing) suite providers and ERPs, with few companies focussed only on spend analysis or broader analytics. In fact, upon review of 25 major players, only Analytics 8 Spend View, Rosslyn Analytics, Sieveo, Spend 360, and SpendHQ remain in the stand-alone best-of-breed spend analytics space. Moreover, when you look at larger analytics focussed enterprises with spend analysis offerings, only Opera Solutions and PRGX stand out as most of the suite providers are still offering last generation or acquired solutions.

And even though there are only a few standalone providers and a few suite providers that stand out, TAMR, whose customers are primarily large Fortune 500 / Global 3000 customers, is in a class almost its own. Many of the standalone providers left are focussed on the mid-market and many of the leading analytics companies, like PRGX that focusses on audit recovery, specialize in other areas of analysis. At the end of the day, only Opera addresses the full range of analytics that TAMR does.

TAMR is relatively unique because, with TAMR, you can start with spend analysis and then deploy the same platform throughout the enterprise and marry marketing and social media impact analytics with spend to analyze results and outcomes per dollar of spend on a campaign basis, collect NPD and innovation challenge data and measure the outcomes from that spend on a fine-grained level, and compare investment opportunities against spend reduction opportunities and see which has the better outcome for the enterprise long term.

Like Opera, TAMR is built around advanced probabilistic machine learning algorithms that can work on any kind of data and can identify probably related and duplicate data in any domain. When human experts label a small subset of data elements, related data elements, and duplicate data elements, the algorithms can quickly adapt to the data sets and classification can occur quickly and accurately.

With regards to spend analysis, TAMR has built out a complete interface to their platform in Tableau that allows an analyst to see what has been classified, drill down, and see confidences in addition to standard supplier groupings, spend by category, spend by supplier, etc. The demo drill-down reporting suite is already more extensive than the standard offering from most of the pure-play spend analysis players and the alternate view into mappings and confidences will be more familiar to Tableau users than TAMR‘s built-in UI.

For a deeper dive into the strengths and weaknesses of this new analytics platform, check out the deep dive by the doctor, the prophet, and the maverick over on Spend Matters Pro (Part I).

SpendHQ: Revving Up Visibility Into Your Supply Base

When we last dug into SpendHQ back in 2014 (Part I, Part II, Part III, and Part IV), we noted how this solution has grown from a simple spend reporting tool into a fully featured spend visibility tool that tracks all of your spend over time — by category, department, and user; a category management tool that lets you dive into category spend and filter down to the items of interest, see managed vs unmanaged spend, and track compliance; and, as of the next release later this quarter, track contract meta data and do basic contract lifecycle management.

We also noted that while it was not the most powerful (ad-hoc) spend analysis solution on the market, it was a really great solution for a mid-market company without a (useable) spend analysis or visibility solution that needed to get one up and running quickly, accurately, and usefully (as the solution has more power and capabilities than the average company needs to get great results). Within 4-6 weeks, a company with no spend analysis capability can be up and running 100% and be making useful, informed decisions.

Since then, they have been hard at work improving the contract module; adding a new compliance module in the visibility engine that allows the user to instantly see, for the selected categories, the addressable spend, the managed spend, the compliance rate, and the impact rate; and a brand new vendor detail module that sits on top of their brand new supplier database that contains information on about 20 Million entities that was formed from the fusing of their database of over 7.5 M entities that they built up over 12 years of operation and InsideView’s database of over 15 Million entities. The database has basic vendor information (address, ownership, status, industry, revenue, etc.), insights (on products, services, strengths, etc.), family tree (which contains ownership, subsidiary and sibling information), and financial data. A user can also see all associated contracts in the contract module and click into the details of each one as required.

One of the gems of the platform is the new and improved Category Management module with greatly enhanced savings management capability. On a category basis, this module summarizes spend, managed spend, core list compliance, and pricing accuracy — where each unit purchased is compared against the contract price. This allows an organization to identify maverick spend and overspend during the contract (on every refresh) and address issues as they arise. Within a category, they can drill into each item and see total spend and drill into spend by location and/or buyer, allowing them to zero in on maverick spend and spend that is priced off-contract. The pricing accuracy can drill down from a category to an item if need be and track inaccuracies, undercharges, overcharges, and overall error rate (as well as overall loss).

In addition, the particular interface customization and support for MRO, T&E, shipping and small parcel spend categories, often overlooked “tail spend”, is far superior to an average product and lets a buyer not only figure out what is maverick or going to on-contract suppliers (but being billed at off-contract rates), but how the spend breaks down across base charges, fuel charges, surcharges, and so on. This allows you to drill into the cost drivers of categories and products, and attack the real cost drivers in a strategic engagement. The specific capabilities built for shipments in particular are quite good. The shipment analytics breaks costs down into accessorials, zones, and fuel surcharges so that an organization can see precisely how the spend is breaking down, where the bulk of the charges are, and where any overspend are.

SpendHQ was built for the sourcing organization that wants a best of breed spend analysis and visibility tool and support maintaining and interpreting it, with the option to engage the right expert at the right time in the right categories to maximize savings. Its more of a “savings as a service” offering than the majority of other spend analysis players, and the best results come from augmenting it with ISG’s sourcing expertise that can help identify the right category to source to maximize savings at any given time. It’s a vendor that should definitely be kept on your radar.

For a deeper dive into SpendHQ, keep an eye out for the upcoming in-depth Spend Matters Pro review [membership required] by the doctor and the prophet that will appear later this summer.