Monthly Archives: March 2015

Just What is “Best Value”, Part Deux!

In yesterday’s post, we discussed an article in a recent edition of Purchasing Tips (by Charles Dominick of Next Level Purchasing) that asked What is Best Value Procurement where he stated that “best value” should be a hard metric measurable in financial terms and expressed in units of currency and not a soft metric where factors other than price are used in determining a supplier and/or product to select for purchase (as that is weighted average supplier/product scoring).

We noted that SI tends to agree, but that there are often issues with trying to assign a(n exact) hard dollar revenue increase or cost decrease to an event that has not yet happened. Even the illustrative example used by Mr. Dominick in trying to choose between machine A and machine B to automate a production line is not cut and dry. For example, if the organization stops manufacturing a product before production line end of life or has the option to lease vs. buy the machine, the calculations get complex. But this is just the beginning.

When it comes to making an IT purchase, the “best value” calculations become a bit of a nightmare. First of all, there is system cost. Depending on whether you want to go with a true SaaS, hosted ASP (which might be wearing a cloud disguise), or on-site hosted solution, as discussed in our classic series on the Enterprise Software Buying Guide (Part V: Cost Model), there are anywhere from four to eleven core up-front and on-going costs that need to be considered (plus ancillary costs for complex or special systems). (And even with the free calculation template provided in the classic SI post on uncovering the true cost of an on-premise sourcing/procurement software solution, the calculation is still a nightmare. How confident are you in the integrator’s estimate? How secure do you feel about the amount of training time (and budget) that will be required? How reliable are the ongoing support level and associated cost calculations.)

Assuming you can work through the system cost equation, which can be quite a doozy (doozy, not doozer, although you will likely need doozer cooperation levels to make any new IT system work these days), you then need to work through the value equation. Just how much value can be expected from the system over the timeframe, and how accurate is that prediction. There are multiple components to this calculation.

  • Throughput Increase
    if the system increases the number of invoices that can be m-way matched, increases the number of sourcing events that can be run, or automates the production of trade documents, this needs to be calculated first as these numbers are need to compute the savings
  • Efficiency Savings
    how much manpower is saved (and how much can therefore be reassigned or eliminated) and how much is the HR expenditure accordingly reduced
  • Cost Savings
    how much cost is expected to be avoided either by increased throughput or the increased performance offered by the system (such as defect reduction, which reduces repair costs)

Obviously, these calculations are not straightforward. In the case of efficiency savings, since every resource (and type) has a different cost (based on salary and associated benefits), the best you will be able to do is estimate an average cost for the manpower by hour (or day). In the cast of cost savings, it’s more than just an industry average, it’s an industry average for a company at a similar stage of competency, with a similar sized workforce, and a similar production or spend pattern. Let’s take spend analysis. If the company is a leader with close to 80% of spend under management, has been sourcing against industry benchmarks, and has used advanced negotiation (and optimization) techniques on high value or key categories (with the help of a third party, if necessary), the company is likely not only aware of its top n categories, but has likely strategically sourced the majority of next n categories as well and the untapped opportunities would represent less than 20% of its spend. This company would only expect to see the industry average 11% savings on roughly 10% of its spend and would likely only see a few percentage points on the spend under management in the current economy. In comparison, if it is an average company only had 45% of its spend under management, had not used advanced sourcing techniques in the past, and only sourced a few categories against benchmarks, it might expect to see the industry average savings of 12% on 40% of its spend and 5% to 6% on the rest. The up-front savings potential (over 1 to 3 years) for this average company on a new spend analysis system would be four times that of the industry leader! It might be the case that the industry leader might need the new system to properly monitor and analyze its spend going forward more efficiently to help it avoid bad decisions in the future, but now we are in cost avoidance territory, and fuzzy territory at that. In hard dollar costs, all one can argue is additional manpower reduction.

And we still haven’t dived to the bottom of the iceberg. In other words, the best definition of best value is a hard dollar metric, but it might be the hardest metric of all to calculate.

Just what is “Best Value”?

In a recent edition of Purchasing Tips over on Next Level Purchasing, Charles Dominick asked What is Best Value Procurement? In the article, he notes that many people use the term “best value procurement” to describe purchasing decisions where factors other than price are used in determining the supplier and/or product to select for purchase and states that he believes that this is “weighted average supplier/product scarring”, which it is.

In his view, value should be measurable in financial terms and expressed in units of currency. I tend to agree, but there are issues with trying to assign a(n exact) hard dollar revenue increase or cost decrease to an event that has not yet happened.

In his illustrative example of choosing between machine A and machine B to automate a production line and reduce the labour needed to keep it running (in an effort to, hopefully, allow the organization to either redeploy the personnel on higher-value tasks or, if not possible, replace those jobs with jobs that could generate more value for the organization down the road), it seems cut-and-dry. Just compute the value-to-cost ratio (where the value, as defined by the estimated labour savings, is divided by the cost of the new machine, which should include purchase, installation, and additional maintenance costs over the expected lifetime). In this case, one machine will generate a higher value-to-cost ratio and that is the machine you should purchase for the organization.

Assuming, of course, that you are sure the machine will have the indicated lifespan and will be useful to you for that lifespan. For example, what happens if you stop making the product in three years but your value calculations are for five years, the expected lifetime of the machine. The value-to-cost calculations will still rank the machines in relative order (as only the value changes), but the return might not look so enticing. And what about the situation where you can instead lease one of the machines from a third party (instead of buying it) and, because that machine in particular is made to a higher quality standard, get an annual lease that is only 1/10th, and not 1/5th, of the purchase cost? In this situation, a machine that cost twice as much would not only have the same value-to-cost ratio but, if you had to sell the machine you bought after three years, the leased machine would have a higher value-to-cost ratio since you’d likely not get the full undepreciated book value for the machine you bought.

And this is just a “best value” calculation on a simple piece of machinery. Consider the difficulty when trying to compute a “best value” on a technology platform purchase, where such platform is intended to improve your sourcing, procurement, supplier relationship management, or similar supply management process. It’s not just up-front cost. It’s implementation. It’s maintenance. It’s operational manpower savings on tactical tasks. It’s efficiency improvements (which have a value in terms of more events or throughput, which translates into generated value) and it’s additional cost reductions identified through the platform (which can be estimated based on benchmarks, but not predicted). How do you do that “best value” calculation? What number do you use? Do you compute a range and use the middle? Do you identify all platforms with a minimum acceptable value-to-cost ratio in terms of guaranteed hard-dollar savings and then select the best-value using the platform with the maximum value-to-cost potential?

There are no easy answers and costs alone don’t always tell the whole story.

Decisions Should be Data-Derived – But They Should Not Be Big Data Driven!

In our recent post where we noted that it’s nice to see CNN run a piece that says big data is big trouble we noted that big data is big danger because more data does not automatically translate into better decisions. Better data translates into better decisions. And often that better data comes in the form of a small set of focussed data. For example, if one is trying to determine the right set of features to include in the next version of a product, the best data points are those that represent the desires of your best current customers who are most likely to buy the product. This is especially true if the most profitable market segment are enterprise business customers that buy thousands of licenses or units. If you only have a few dozen of these customers, these few dozen data points are more relevant than thousands of data points you’d get from a mass-market survey which would likely include hundreds of data points from customers who are only vaguely interested in your product (and who would likely never buy it).

Data does matter. But only the right data matters. That’s why only companies in the top-third of their industry in the use of data-driven decision making are 5% more productive and 6% or profitable than their competitors (as per “an introduction to data-driven decisions”. If it was just a matter of lots of data, then all companies would be more productive and half would be noticeably more profitable than their peers.

So how do you know if the data is good? Ask the right questions. In the HBR piece, the author lists six key questions that should be asked before acting on any data:

  1. What is the data source?
  2. How well does the data sample represent the population?
  3. Does the data distribution include outliers? Do they affect the results?
  4. What assumptions are behind the analysis? Are there conditions that would render the assumptions and model invalid?
  5. What were the reasons behind selecting the data and approach?
  6. How likely is it that independent variables are actually causing changes in the dependent variable?

And the answers that are received should be relevant to the problem at hand. For example, if we go back to our software / hand-held device example, the answers received should be along the lines of:

  1. Business Customer Surveys
  2. Over 70% of the organization’s largest accounts are represented
  3. Some small customers are included as well, but they are less than 10% of respondents and do not affect the results
  4. The assumptions are that the largest accounts provide the most relevant data. Currently, major account satisfaction is good and the data can be relied on so there are no current conditions that would affect assumptions.
  5. Large corporate customers represent over 60% of the company’s profit, so focussing on their needs first was the rationale.
  6. The surveys were designed to minimize the impact of independent variables, so the likelihood is low.

In this situation, you know the data is good, the approach is good, and the assumptions are relatively sound and you can likely count on the results. And, more importantly, the organization should act on them because it’s likely that any frequent correlation in the data indicates a causal hypothesis (if you add the indicated features, then the current customer base will buy the next version) and the benefits outweigh the risk (as a sufficient sales volume will cover the R&D costs).

And, just like the HBR article says, you don’t even have to like math to make the right decision. (Although there’s no reason not to like math.)

It’s Nice To See CNN Run a Piece that Says Big Data is Big Trouble

the doctor doesn’t like the phrase “big data” or the “big data” craze. First of all, as he has said time and time again, we’ve always had more data than we could process on a single machine or cluster and more data than we could process in the time we want to process it in. Secondly, and most importantly, just like the cloud is filled with hail, big data is filled with big disasters waiting to happen.

As the author of the article on on the big dangers of ‘big data’ astutely points out, there are limits to the analytic power of big data and quantification that circumscribe big data’s capacity to drive progress. Why? First of all, as the author also points out, bad use of data can be worse than no data at all. As an example, he cites a 2014 New York Times Piece on Yahoo and it’s Chief Executive which demonstrated the unintended consequences of trying to increase employee drive and weed out the chaff by way of scorecard-based quarterly performance reviews which limited how many people on a team could get top ratings. Instead of promoting talent and driving talented people together, it split them up because, if you were surrounded by under performers, you were sure to get the top score – but if you were surrounded by equals, you weren’t.

This is just one example of the unintended consequences of trying to be too data driven. Another example is using average call time in a customer support centre versus number of calls to close a ticket as a measure of call centre agent performance. If an agent is measured on how long she spends on the phone on average, she is going to try to take shortcuts to solve a customer’s problem instead of getting to the root cause. For example, if your Windows PC keeps locking up every few days and a re-boot fixes it, you will be told to proactively reboot every 24 hours just to get you off the phone. But that doesn’t necessarily fix the problem or guarantee that you will not have another lock-up (if the lock-up is a certain combination of programs opened at the same time that refuse to share a peripheral device, for example). As a result, the customer will end up calling back. Or, if she can’t solve your problem, you will be switched to another agent who “knows the system better”. That’s poor customer support, and all because you’re keeping track of the average time of every call and computing averages by rep and department.

Big data will let us compute more accurate economic forecasts, demand trends, process averages, and so on, but, as the author keenly points out, many important questions are simply not amenable to quantitative analysis, and never will be. The examples of where your child should go to college, how to punish criminals, and whether or fund the human genome project are just a few examples. Even more relevant are product design queries. 34% of users want feature A, 58% want feature B, and 72% want feature C, but how many want features A and B or A and C or B and C or all three features? And how many will be put off if the product also contains a feature they don’t want, is too confusing due to too many frivolous features, or doesn’t have all important feature D that you didn’t ask about, but now have to have because your competitor does?

And, even more important, McKinsey, which in 2011 claimed that we are on the cusp of a tremendous wave of innovation, productivity and growth … all driven by big data had to recently admit that there is no empirical evidence of a link between data intensity … and productivity in specific sectors. In other words, despite all of the effort put into big data projects over the last few years, none have yielded any results that are conclusively beyond results that would have been achieved without big data.

And, most importantly, as someone who has studied chaotic dynamical systems theory, the doctor can firmly attest to the fact that the author is completely correct when he says understanding the complexity of social systems means understanding that conclusive answers to causal questions in social systems will always remain elusive. We may be able to tease out strong correlations, but correlation is not causation. (And if you forget this, you better go back and take another read through Pinky and the Brain’s lesson on statistics.)