This is an update of a post that originally ran way back in 2007. Yes, two, double-o seven. Seventeen years ago. It is being updated because
- it needs a re-posting
(as very few of you will find it that deep in the archives) - most of the vendors originally mentioned are gone
However, if you read, and remember, the original, you’ll realize that, like my article where the doctor goes mental on optimization myths (which was recently shared on LinkedIn), it doesn’t need much updating and what was written seventeen years ago is still valid to this day. (When you write to inform vs. to create meaningless buzz, it really does stand the test of time.) Let’s begin.
Not all optimization vendors are equal … and, more importantly, not all vendors that claim to have strategic sourcing decision optimization (SSDO) actually have it (since the underlying algorithms and model needs to meet a stringent set of requirements to be true SSDO), with some systems, to this day, barely qualifying as decision support. Thus, since the need for optimization is as desperate as it has ever been with costs again skyrocketing, risks rising rapidly, carbon control being critical, and supply assurance necessary for sustained operations, it’s time to make sure you know how to qualify a potential provider. This means you need to not only understand the basics of what SSDO does (see the archives), but also how to distinguish between the relative strengths and weaknesses of the different offerings, as well as how much strength you really need.
You need to buy optimization at the strength, and usability level, that you need — especially if the vendor is pricing it according to its power, or computational requirement. And while there is no such thing as too much, the reality is that a 95% solution is often more than enough as the entire point is understanding the optimal solution against each dimension (cost, risk, carbon), the cost of compromise between the trade-offs, and the cost of going with a preferred, versus calculated, vendor award. And doing this for EVERY sourcing event. Once you factor in enough discounts and constraints, it’s almost impossible to calculate the best award in a spreadsheet, and the insight of what you could be spending, versus what you are, how low your risks could be, versus what they are, and how much you could alter your carbon footprint, vs what your footprint is today, is invaluable. Even if you never select a recommended solution, the key is understanding how good your (preferred) award actually is.
Before we get to the (starting) question list, it should be pointed out that it’s almost impossible to cover every question, as many of the questions you should be asking depend on the answers you receive to your first few questions, but the question list below is a good starting point.
1. Does the product meet the four criteria for strategic sourcing decision optimization?
- Sound & Complete Mathematical Foundations : such as MILP solutions based on simplex, branch and bound, and interior point algorithms as many simulation, heuristic, and “AI” algorithms DO NOT guarantee analysis of every possible solution (sub)space given enough time, and, thus, are not “complete” in mathematical terms (and if they incorporate Gen-AI, they aren’t even “sound” in that they may not even compute an award that satisfies the constraints!)
- True Cost Modelling :
that supports tiered bids, discounts, and fixed cost components — the model must be capable of supporting all of the bid types being collected, as well as the cost breakdowns - Sophisticated Constraint Analysis : at a minimum, the model must be able to reasonably support generic and flexible constraints in each of the following four categories
- Capacity / Limit: allowing an award of 200K units to a supplier who can only supply 100K units does not make for a valid model
- Basic Allocation: you should be able to specify that a supplier receinves a certain amount of the business, and that business is split between two or more suppliers in feasible percentage ranges
- Risk Mitigation: you should be able to force multiple suppliers, geographies, lanes, etc. to mitigate those risks without specifying specific suppliers, geographies, lanes, etc. to take advantage of the full power of decision optimization
- Qualitative: A good model considers quality, defect rates, waste, on-time delivery, etc., and must support qualitative factors and minimum and average scores across the award
- What-If? Capability : The strength of decision optimization lies in what-if analysis. Keep reading.
2. Does it support the creation of multiple what-if scenarios per event?
Furthermore, does it simplify the creation of these scenarios? The true power of decision optimization does not lie in the model solution, but the ability to create different models that represent different eventualities (as this will allow you to hone in on a robust and realistic solution), to create different models off of a base model plus or minus one or more constraints (as this will help you figure out how much a business rule or network design constraint costs you), and to create models under different pricing scenarios (to find out what would happen if preferred suppliers decreased prices or increased supply availability).
3. How fast is it for different average model sizes?
And can performance be tweaked? Optimization takes what it takes. That being said, if one solution takes an average of 1 hour for an average scenario, and another solution takes 10 minutes, all things being equal, if you have compressed sourcing cycles, the 10 minute solution might be better. Emphasis on “might”. This is only true if the faster solution is of the same quality – some models, and some solvers, sacrifice quality and accuracy for speed. The best solution will let you trade off “tolerance” and accuracy for speed. Sometimes it’s easy to get within 1% or 2% in a few minutes, even though that last 1% or 2% could take hours. On a model with low total savings potential, getting within 1% may be enough. And when trying to hone in on the right what-if scenario, it’s nice to get within 1% quickly and then allow the right scenario to run to completion over lunch (or if its a huge model, over night) after you’ve quickly analyzed half-a-dozen scenarios and settled on your preferred scenario. Thus, tweaking ability is very important.
4. Is it “true” real-time or “near” real-time?
Thanks to significant advances in processor and hardware performance as well as off-the-shelf optimizer technology (like IBM ILog’s CPlex), it’s now possible to rapidly re-build and re-solve even very large models using off-the-shelf modeling languages in seconds, allowing for e-auction tools that keep the model relatively moderate in comparison, and presolve with seed bids (current prices, market prices, last quotes), to incorporate decision optimization in real-time by simply updating a few parameters and re-solving the model every (few) parameter(s) update (depending on model-size) on a high-powered multi- core server with an appropriately configured and optimized solver (which can spin off copies and have each processor work on a different subspace). However, if the approach the product takes is to rebuild and resolve the model on every update, that’s not real-time, that’s near real time, and the slowdown could be significant for large models. (To clarify further, real-time optimization requires the ability to merge model construction and model solution in such a way that a new bid can be introduced as a parameter change that does not require the optimizer to rebuild the sparse model matrix and start the solution process over from scratch.)
5. Can you describe two or three scenarios you have encountered where you could not model the situation exactly?
And, more importantly, how did you work around the issue, and how accurate was the final result. The real world is messy, compared to models that are clean, only so much data is available, and math can only model as much as the minds who created the model could conceive. As a result, no optimization model can handle every real-world scenario 100% accurately. If a vendor representative says so, he’s either lying through his teeth or not competent enough to be selling the product. (Note that: I’ll have our optimization expert get back to you on that is a good answer from an average sales representative.) This is about the only way to get a decent idea of how appropriate the tool is for you. If the scenarios were complex and the constraints based on business rules you hardly ever, or never, use, then the solution is probably okay for you. If the scenarios were simple and the constraints based on business rules you use all the time, it’s probably not the tool for you.
6. Would you be willing to demo your solution to, and answer questions from, our consultant who understands both our needs and decision optimization technology?
Let’s face it -– just like the right decision optimization tool can deliver huge savings multiples on your investment (10X or more), the wrong tool will simply represent a six (or seven) figure cost that yields little return. If you can’t tell the difference, and there’s no shame in admitting you can’t if you’ve never used this type of technology before, then you should bring in a consultant who can to help you select the right technology, and ensure you are appropriately trained on it, until you are self sufficient and saving an average of 10% or more per project put through the tool.
7. Can we do a pilot project at-cost (or gain-share) before committing to a long term license?
If you like what you hear, but are still unsure, or are having problems getting the budget approved, a pilot is often the way to go! (Note that I did not use the word “free”!) If you’re not willing to sign a license, given the sophistication of this technology and the amount of effort the provider is going to have to allocate to support you through the pilot and ensure you are successful, you need to be willing to pay for services at a rate that is sufficient to cover the provider’s cost for the pilot -– especially considering that many of the companies that offer affordable optimization offerings are only able to do so because they keep their costs and overheads down.