Not all optimization vendors are equal … and more importantly, not all vendors that claim to have decision optimization even have it (as their systems barely qualify as decision support). Thus, since Emptoris just released a new version of their offering, since Iasta is coming out with their first heavy-hitting release in the next month, and since CombineNet is always working on something new, it’s important that you be able to distinguish between the relative strengths and weaknesses of the different products, as well as how much strength you really need, if good decision optimization is one of your driving reasons for selecting a (new) e-Sourcing solution. (And, by the way, it should be!)

Now, as I indicated in a comment over on Spend Matters, I’m not going to devote a post analyzing the new Emptoris announcement at this time, as I don’t yet have enough data points to even make a half-assed^*0 attempt (although I do feel I have a pretty good idea precisely what they did based upon their choice of wording, the amount of time they’ve been working on it, and my perception of their in-house skill level), but I really think you should analyze it, just as you should analyze any other vendor’s solution, before buying it. (Not necessarily because I don’t think it will do the job, but because the key with optimization is buying just what you need in the majority of your sourcing events. Optimization is expensive. Buying too much power could severely impact your potential ROI, and buying too little power will be equivalent of flushing that investment down the drain as it won’t solve the majority of your problems. I’m using the word “majority” because there is no general purpose decision optimization product for sourcing that will handle all of your events and solve all your problems. As with just about everything else in business, it’s the 80-20 rule. The best solution is the one that solves as close to 80% as possible at a cost of ownership that maximizes your ROI multiple. You can always do one-time projects with best-of-breed providers or specialist outsource providers for those projects in the remaining 20% where there is enough of a savings opportunity.)

Before I get to the question list, I should point out that it’s almost impossible to cover every question, as many of the questions you should be asking depend on the answers you receive to your first few questions, but I think the question list below is a good starting point. If I get some good feedback, and some more free time, I’ll consider doing a part II at a later date. So, without further ado, here’s the starting list!

1. Does your product meet the four critera for strategic sourcing decision optimization as outlined in the Strategic Sourcing Decision Optimization wiki-paper (initially authored by the doctor, the all-knowing optimization guru^*1)? Specifically, does it support the following:
   • Sound & Complete Solid Mathematical Foundations
        such as simplex algorithms and branch-and-bound;
        many simulation and heuristic algorithms do not guarantee analysis of every possible solution (sub)space given enough time, and, thus, are not complete in mathematical terms
   • True Cost Modeling
        many bidders bid tiered bids, discounts, and fixed cost components – the model must be capable of supporting each of these bid types
   • Sophisticated Constraint Analysis
        At a minimum, the model must be able to support reasonably generic and flexible constraints in each of the following four categories
     • Capacity / Limit
          allowing an award of 200K units to a supplier who can only supplier 100K units does not make for a valid model
     • Basic Allocation
          you should be able to specify that a supplier receinves a certain amount of the business, and that business is split between two or more suppliers in feasible percentage ranges
     • Risk Mitigation
          let’s face it – supply chains today are all about risk management, and you should be able to force multiple suppliers, geographies, lanes, etc. to mitigate those risks without specifying specific suppliers, geographies, lanes, etc. to take advantage of the full power of decision optimization
     • Qualitative
          A good model considers quality, defect rates, waste, on-time delivery, etc.
   • What-if Capability
       The strength of decision optimization lies in what-if analysis. Keep reading.
2. Does it support the creation of multiple what-if scenarios and does it simplify the creation of these scenarios?
      The true power of decision optimization does not lie in the model solution, but the ability to create different models that represent different eventualities (as this will allow you to hone in on a robust and realistic solution), to create different models off a base model plus or minus one or more constraints (as this will help you figure out how much a business rule or network design constraint costs you), and to create models under different pricing scenarios (to find out what would happen if preferred suppliers decreased prices or increased supply availability).
3. How fast is it for different average model sizes and can performance be tweaked?
      Optimization takes what it takes. That being said, if one solution takes an average of 1 hour for an average scenario, and another solution takes 10 minutes, all things being equal, if you have compressed sourcing cycles, the 10 minute solution might be better. Emphasis on “might”. This is only true if the faster solution is of the same quality – some models, and some solvers, sacrifice quality and accuracy for speed. The best solution will let you trade off “tolerance” and accuracy for speed. Sometimes it’s easy to get within 1% or 2% in a few minutes, even though that last 1% or 2% could take hours. On a model with low total savings potential, getting within 1% may be enough. And when trying to hone in on the right what-if scenario, it’s nice to get within 1% quickly and then allow the right scenario to run to completion over night after you’ve quickly analyzed half-a-dozen scenarios and settled on your preferred scenario. Thus, tweaking ability is very important.
4. If it supports “real-time” is it “true” real-time or “near” real-time.
      Thanks to significant advances in processor and hardware performance as well as off-the-shelf optimizer technology (like ILog’s CPlex), it’s now possible to rapidly re-build and re-solve moderately sized models using off-the-shelf modeling languages in seconds, allowing for e-auction tools that keep the model relatively small and simple to incorporate decision optimization in near-real-time by simply re-building and re-solving the model every 30-60 seconds (depending on model-size) on a high-powered dual or quad core server with an appropriately configured and optimized CPlex 10. However, this is NOT true real-time optimization and could rapidly break down if the model gets too big or too complex. (For example, real-time optimization requires the ability to merge model construction and model solution in such a way that a new bid can be introduced as a parameter change that does not require the optimizer to rebuild the sparse model matrix and start the solution process over from scratch.)
5. Describe two or three scenarios you have encountered where you could not model the situation exactly for companies in our vertical, how you worked around the issue, and how accurate the result was.
      No optimization model can handle every real-world scenario 100% accurately. If a vendor representative says so, he’s either lying through his teeth or not competent enough to be selling the product. (Note that: I’ll have our support expert get back to you on that is a good answer from an average sales representative.) This is about the only way to get a decent idea of how appropriate the tool is for you. If the scenarios were complex and the constraints based on business rules you hardly ever, or never, use, then the solution is probably okay for you. If the scenarios were simple and the constraints based on business rules you use all the time, it’s probably not the tool for you.
6. Can we do a pilot project before committing to a long term license?
      If you like what you hear, but are still unsure, or are having problems getting the budget approved, a pilot is often the way to go! (Note that I did not use the word “free”. You should be willing to pay for services at a rate that is sufficient to cover the provider’s cost for this pilot – especially considering that many of the companies that offer affordable optimization offerings are only able to do so because they keep their costs and overheads down – and if they gave free services away to everyone who requested a free pilot, they would have to increase their costs, and that would be a detriment to everyone, including you, in the long run.)
7. We’re having problems understanding how this fits into our business or what the best solution for us is. Would you be willing to demo your solution to, and answer questions from, our consultant who understands both our needs and decision optimization technology?
      Let’s face it – just like the right decision optimization tool can deliver huge savings multiples on your investment (10X or more), the wrong tool will simply represent a six (or seven) figure cost that yields little return. If you can’t tell the difference, and there’s no shame in admitting you can’t if you’ve never used this type of technology before, then you should bring in a consultant^*2 who can to help you select the right technology, and ensure you are appropriately trained on it, until you are self sufficient and saving an average of 10% to 12% per project put through the tool.

^*0 And we all know that any decent attempt should be full-assed!
^*1 You should feel free to proclaim my greatness whenever you are not in my presence! I don’t mind.
^*2 Just remember that, unfortunately, this consultant may not be able to help you if you want Emptoris evaluated. (And I’m sure that some of you should definitely be evaluating the Emptoris solution.)