Not all optimization vendors are equal … and more importantly, not all vendors that claim to have decision optimization even have it (as their systems barely qualify as decision support). It’s important that you be able to distinguish between the relative strengths and weaknesses of the different products, as well as how much strength you really need, if good decision optimization is one of your driving reasons for selecting a (new) e-Sourcing solution. (And, by the way, as I have been stating since day one, it should be!)

Before I get to the question list, I should point out that it’s almost impossible to cover every question, as many of the questions you should be asking depend on the answers you receive to your first few questions, but I think the question list below is a good starting point.  So, without further ado, here’s the basic list!

1. Does your product meet the four critera for strategic sourcing decision optimization as outlined in the Strategic Sourcing Decision Optimization wiki-paper (initially authored by the doctor, the all-knowing optimization king*1)? Specifically, does it support the following:
• Sound & Complete Solid Mathematical Foundations
such as simplex algorithms and branch-and-bound;
many simulation and heuristic algorithms do not guarantee analysis of every possible solution (sub)space given enough time, and, thus, are not complete in mathematical terms
• True Cost Modeling
many bidders bid tiered bids, discounts, and fixed cost components – the model must be capable of supporting each of these bid types
• Sophisticated Constraint Analysis
At a minimum, the model must be able to support reasonably generic and flexible constraints in each of the following four categories

• Capacity / Limit
allowing an award of 200K units to a supplier who can only supplier 100K units does not make for a valid model
• Basic Allocation
you should be able to specify that a supplier receinves a certain amount of the business, and that business is split between two or more suppliers in feasible percentage ranges
• Risk Mitigation
let’s face it – supply chains today are all about risk management, and you should be able to force multiple suppliers, geographies, lanes, etc. to mitigate those risks without specifying specific suppliers, geographies, lanes, etc. to take advantage of the full power of decision optimization
• Qualitative
A good model considers quality, defect rates, waste, on-time delivery, etc.
• What-if Capability
The strength of decision optimization lies in what-if analysis. Keep reading.
2. Does it support the creation of multiple what-if scenarios and does it simplify the creation of these scenarios?
The true power of decision optimization does not lie in the model solution, but the ability to create different models that represent different eventualities (as this will allow you to hone in on a robust and realistic solution), to create different models off a base model plus or minus one or more constraints (as this will help you figure out how much a business rule or network design constraint costs you), and to create models under different pricing scenarios (to find out what would happen if preferred suppliers decreased prices or increased supply availability).
3. How fast is it for different average model sizes and can performance be tweaked?
Optimization takes what it takes. That being said, if one solution takes an average of 1 hour for an average scenario, and another solution takes 10 minutes, all things being equal, if you have compressed sourcing cycles, the 10 minute solution might be better. Emphasis on “might”. This is only true if the faster solution is of the same quality – some models, and some solvers, sacrifice quality and accuracy for speed. The best solution will let you trade off “tolerance” and accuracy for speed. Sometimes it’s easy to get within 1% or 2% in a few minutes, even though that last 1% or 2% could take hours. On a model with low total savings potential, getting within 1% may be enough. And when trying to hone in on the right what-if scenario, it’s nice to get within 1% quickly and then allow the right scenario to run to completion over night after you’ve quickly analyzed half-a-dozen scenarios and settled on your preferred scenario. Thus, tweaking ability is very important.
4. If it supports “real-time” is it “true” real-time or “near” real-time.
Thanks to significant advances in processor and hardware performance as well as off-the-shelf optimizer technology (like ILog’s CPlex), it’s now possible to rapidly re-build and re-solve moderately sized models using off-the-shelf modeling languages in seconds, allowing for e-auction tools that keep the model relatively small and simple to incorporate decision optimization in near-real-time by simply re-building and re-solving the model every 30-60 seconds (depending on model-size) on a high-powered dual or quad core server with an appropriately configured and optimized CPlex 10. However, this is NOT true real-time optimization and could rapidly break down if the model gets too big or too complex. (For example, real-time optimization requires the ability to merge model construction and model solution in such a way that a new bid can be introduced as a parameter change that does not require the optimizer to rebuild the sparse model matrix and start the solution process over from scratch.)
5. Describe two or three scenarios you have encountered where you could not model the situation exactly for companies in our vertical, how you worked around the issue, and how accurate the result was.
No optimization model can handle every real-world scenario 100% accurately. If a vendor representative says so, he’s either lying through his teeth or not competent enough to be selling the product. (Note that: I’ll have our support expert get back to you on that is a good answer from an average sales representative.) This is about the only way to get a decent idea of how appropriate the tool is for you. If the scenarios were complex and the constraints based on business rules you hardly ever, or never, use, then the solution is probably okay for you. If the scenarios were simple and the constraints based on business rules you use all the time, it’s probably not the tool for you.
6. Can we do a pilot project before committing to a long term license?
If you like what you hear, but are still unsure, or are having problems getting the budget approved, a pilot is often the way to go! (Note that I did not use the word “free”. You should be willing to pay for services at a rate that is sufficient to cover the provider’s cost for this pilot – especially considering that many of the companies that offer affordable optimization offerings are only able to do so because they keep their costs and overheads down – and if they gave free services away to everyone who requested a free pilot, they would have to increase their costs, and that would be a detriment to everyone, including you, in the long run.)
7. We’re having problems understanding how this fits into our business or what the best solution for us is. Would you be willing to demo your solution to, and answer questions from, our consultant who understands both our needs and decision optimization technology?
Let’s face it – just like the right decision optimization tool can deliver huge savings multiples on your investment (10X or more), the wrong tool will simply represent a six (or seven) figure cost that yields little return. If you can’t tell the difference, and there’s no shame in admitting you can’t if you’ve never used this type of technology before, then you should bring in a consultant who can to help you select the right technology, and ensure you are appropriately trained on it, until you are self sufficient and saving an average of 10% to 12% per project put through the tool.