Category Archives: Forecasts

Forecasting Flux

Twenty years ago, we were sure to tackle forecasting and all the potential failures that could arise from the selection of the wrong method or the misapplication thereof in one of our first forecasting posts where we contrasted the two primary techniques in use at the time: judgemental forecasts and statistical forecasts.

In summary, we noted that:

Forecast Type Strengths Weaknesses
Judgemental * Environmentally Responsive
* “Inside” Information
* Responsive to “One-time” Events
* Cognitive Limitations
* Biases
Statistical * Objective
* Consistent
* No Data Limits
* Considers All Variables
* No Real Time Reaction
* Dependent on Model Formulation
* Not Friendly to “Soft” Information
* Technical Understanding Required

The methodology that was typically used in practice was that a statistical model was run and then an expert adjusted it to their liking. Now, this usually worked okay if the organization had a real expert and access to up-to-date soft data, but otherwise, it was a shot-in-the-dark and poor forecasters would saddle the organization with stock-outs (and the ever-present 8% stockout rate) or excess inventory that would have to be fire-sold in the best case, and simply disposed of in the worst case. (Which is very costly from multiple angles, especially if you have to bury 700,000 cartridges in a landfill, which is where Atari’s E.T. found home in 1983.)

Things have changed in the last two decades. Now the average forecaster has two more options, which we really need to discuss: AI and Blended.

Forecast Type Strengths Weaknesses
AI * flexible model selection
* flexible solution methodology
* flexible interpretation of results
* random model selection
* performance limitations
* hallucinations
Blended * dependable models
* multiple scenarios
* balanced solutions that minimize downsides
* computation time
* data requirements
* scenario weighting

The AI option in particular is really concerning. Given that Gen-AI is, at best, artificial idiocy, you have no idea if the model it selects is appropriate, and even if the model it selects is reasonable, AI has never outperformed the classical statistical model equivalent, and then when you ask it for final interpretation, it can sometimes get the interpretation completely wrong. Generally speaking, modern (Gen) AI worsens forecasts.

In addition, the fact that most S&OP platforms still aren’t using balanced blended models is equally concerning. Considering that taking multiple models into account to produce a blended forecast reduces the chance of (significant) stockout and overstock when the reality doesn’t match the most likely scenario, blended should be the first choice. Especially since it’s better to be a few units short than have 10,000 extra or a few months of extra inventory that will eventually sell then be without for three months (due to long lead times).

In fact, the best method is blended models guided by human judgement that can identify the most likely scenarios, validate the best models, and then let the raw computational power of todays platform compute a solution that is near optimal across all of the different forecasts, rather than over optimizing for the wrong forecast. When you don’t know what the actual demand will be, it’s better to go with a blended solution that might be 2% off of optimal than pick the wrong scenario and end up with an “optimized” forecast that is 20% off.

Despite all of our advances, forecasting is still as much art as science, and only expert human judgement on blended models will produce sound forecasts in today’s rocky economic environment.

Is it time for Supply Driven Demand Creation?

For the past two decades, the goal was JIT DDS — Just in Time Demand Driven Supply — and the ultimate in Sourcing and Procurement was sourcing and procuring to optimize that mode of operation as it minimized costs and maximized revenue. In theory at least. And for a while, more often than not, it did in practice.

When predictability and reliability is high on both the customer side and the supply side, JIT DDS is the ultimate source-to-supply goal. Optimizing delivery time (JIT) minimizes inventory cost and waste, allows shipments to be right sized, and makes cashflow more predictable as spend appropriately correlated to sales. Accurate forecasting (DDS) optimizes sourcing events, minimizes stock-outs, and minimizes waste.

But what happens when:

  • customer demand is unpredictable due to rapid changes in inflation, employment, tarrifs, and other factors
  • supply lines get caught off with little to no warning thanks to strikes, pandemics, wars, and droughts
  • raw materials get scarce or suddenly become unavailable

Supply Stops. Then sales stop. Everything breaks. And there’s no quick fix.

Now, that’s why you have visibility, monitoring, multiple sources of supply, mitigation plans, etc. That works when there are alternate sources, alternate routes, and enough time to react — but when the sources depend on the same raw material or rare component, and that becomes unavailable, one or more of your product lines are brought to a halt. And there’s nothing you can do.

Except for preventing that situation from ever happening in the first place.

The only way to do that is to ensure you are not dependent on raw materials or rare components that could disappear literally overnight.

Which means that if you revenue is heavily dependent on products (and categories) that rely on these raw materials, you need to do your best to identify alternates wherever possible and if those alternates don’t allow you to make roughly the same product, you’ll have to work with marketing and sales to shift demand based on available supply.

In some categories, that won’t be possible — you can’t make certain electronics without rare earths, but you can shift to product designs that minimize them. Depending on the rare earth in question, any change in quantity will affect the design options — not just in material composition, but in size, performance, and cost. If the cost goes up, or performance goes down, it could be difficult to convince the market an alternate design or product is better, but if there are no other real options, that’s what has to happen.

Which is why success in turbulent times is dependent on supply driven demand — and, more importantly, an abandonment of JIT to balanced inventory where enough supply is kept on hand to survive the average delay expected by an interruption, with a buffer.

Roughly Half a Trillion Dollars Will Be Wasted on SaaS Spend This Year and up to One Trillion Dollars on IT Services. How Much Will You Waste?

Before we continue, yes, that is TRILLION, numerically represented as 1,000,000,000,000, repeated twice in the title and yes we mean US (as in United States of America) dollars!

Gartner projects that IT spend will surpass 5 Trillion this year. When you consider that 30% of IT spend is usually for software, and that one third (or more) of software spend is wasted (for unused licenses, which is why we have a whole category of IT and SaaS specialists that analyze your out-of-control SaaS and software spend and typically find 30% to 40% overspend in a few days), that means that roughly half a trillion dollars will be wasted on software this year.

Even worse, Gartner projects that spending on IT Services will reach 1.5 Trillion. And the waste here could be two thirds! Now, we all know that you need IT services to implement, integrate, and maintain those IT systems you buy. But how much do you need? And how much should you pay? Consider that an intermediate software developer should be making 150K a year (or 75/hour), that says that an intermediate implementation specialist shouldn’t be making any more than that, and not billed at more than 3 times that (or 225/hour). But how much are you being billed for relatively inexperienced implementation consultant, with maybe a few years of overall experience and maybe six months on the system that you are installing? the doctor knows that rates of $300 to $500 are not uncommon for these resources that are oversold and overcharged for.

But this isn’t the worst of it. As per our upcoming article Fraud And Waste Are Not The Same Thing, many implementation “partners” will try to get all they can get and make sure that when you go in for a penny, you go in for a pound and they will push for:

  • frequent change orders during implementation, usually billed at excessively high day rates as they have to “divert resources” or “work overtime”
  • unnecessary customizations or real-time integrations that are an extensive amount of work (and cost) when out-of-the-box or daily flat-file synchs are more than sufficient
  • extensive “process evaluation” or “process transformation” processes that are well beyond what you need to eat up consulting hours
  • extensive “best practice” education when your practices are good enough for now and/or those best practices are already encoded in the system you just bought and paid a pretty penny for and just following the default process gives you the same education

That will often double to triple the cost. But that’s not the worst of it. As per comments the doctor has made on LinkedIn, he regularly hears stories of niche providers losing 200K deals because customers said their quote was too low because all the Big X companies quoted over 1,000K for what should be 100K worth of work in their view (and, right or wrong, if a niche firm comes in less with a detailed proposal, they should be evaluated — maybe the Big X, with a very general request, over estimated your requirements and the effort, or maybe the niche firm completely underestimated it — how will you know if you don’t evaluate all the responses?). Literally. This is because, as the doctor has noted in previous posts and comments on LinkedIn:

  • they don’t have always have the talent in advanced tech (and even The Prophet has noted their lack of talent in areas of advanced tech in multiple LinkedIn posts, though he has been much more diplomatic than the doctor in discussing their lack thereof; but he did note in a 2024 advice post that consultancies are going to have a hard time attracting talent this year) — for every area, an average firm will have a team leader who’s a superstar, two or three handpicked lieutenants who are above average, and then 20 to 40 benchwarmers who are junior and not always worth the rate they are charging);  now, as with every general observation, there are exceptions (with some Big X recently acquiring a number of best-in-class technology, analytics, and AI vendors that give them top-notch world class talent, and others actively recruiting top talent form the best tech firms, but every firm is different, and, most importantly, every need is different — it’s up to you to fully qualify your need, review the proposal carefully, and vet the proposed talent, otherwise, it’s your fault if you overpay, fail miserably, and don’t get value
  • some of these firms have an incredible overhead — they got big in good times and built posh offices to house the partners making more than top lawyers who have a lifestyle to maintain (or, in some cases, they just acquired expensive real estate in premiere locations)
  • they don’t always have the knowledge of, or experience in, modern tools — some of which are ten times more powerful than last generation tools; this, of course, means that, in these situations, Big X benchwarmers are using last generation tools which take ten times the manual labour to extract value from
  • etc.

Unless you want to pay 1K an hour, at some of these firms, you’re not guaranteed getting that one superstar resource trying to be the front end to two dozen projects that his three lieutenants are trying to manage, all of which are staffed by junior to intermediate individuals who can barely follow the three to five year old playbook.   (While if you chose a different Big X firm that just acquired a whole consultancy with dozens of top analysts, it’s a different story.)

There’s a reason that The Prophet predicted in his 9th prediction that SaaS Management Solutions [will] Start to Eat Services Procurement Tech and that many companies will go in house if they have tech expertise. Because he realizes that these consultancies will have a hard time not only hiring, but retaining, tech talent when they have hiring freezes, salary freezes, and reduced engagements as more and more companies can’t afford the ridiculous rates they’ve been charging recently. (Companies may not have had a choice during COVID where it was implement on-line collaboration and B2B tech or perish, but now they do.)

But there are still many companies who will, when they encounter a (perceived) tech need, immediately pick up the phone and call their favorite Big X firm and bring them in to help them understand who to bring in for an engagement, instead of widening the net to niche providers who might be 3 to 5 times cheaper, and who will deliver results at least as good, if not better, or, if their proposals won’t cut it, will validate when that multi-million proposal is a great value and will deliver the expected ROI.

Now, again, the doctor would like to stress that, despite how much he insists they are usually not the right solution for specialist advanced tech implementations that aren’t the enterprise systems and suites they usually implement, that Big X are not all bad, and sometimes worth many times more than the high fees they charge. [See when should you use Big X?] Most of these companies started off as management/operational/finance/strategy consultants and grew big because they were one of the best, and in certain domains, each of these companies still are. As they grew, they added more areas and became experts in those.  But no company can, and should, be expected to be an expert in everything!

And while there will be exceptions to the rule (as every one of these companies has some tech geniuses), the reality is that when you need more bodies than there are talented bodies in an entire industry, you’re not going to get them and, because consultancies are not cool when you want to be a tech superstar (and join a startup that becomes a unicorn), the ratio of superstar to above average to average to below average talent in these organizations is much thinner than in multinational tech companies (like Alphabet, Apple, Meta, Microsoft, etc.)  (Because if they were the best of the best, there’s no way they’d lay off 10,000 employees at a time every time the market jitters.)

In short, manage that IT services spend carefully, or you’ll be double paying, triple paying, or worse and providing a big chunk of the roughly ONE TRILLION DOLLARS in IT services overspend that the doctor predicts will happen (again) this year. (Unless, of course, you agree with Doctor Evil who says, why make trillions when we could make … billions. Because that’s exactly what happens when you overpay for software and services. Don’t expect the Big X or Mid-Market to say anything as they get the majority that overspend, and that’s how they stay so profitable.  Plus, they usually need those revenues to deliver what you’re asking for, as ill-defined projects mean they need to make a lot of assumptions and often over engineer to decrease the chance you will be disappointed in the result!  In other words, if you overpay due to your lack of research and preparation, it’s on you. )

How Not to Excel at Forecasting

This post originally ran four years ago. But since a critical mistake is still being made, it’s time for a repost.

How Not to Excel at Forecasting?

Simply put, use Microsoft Excel. It’s appalling that a survey by ToolsGroup and the Global Market Development Centre (GDMC) found that even though two-thirds of companies in the consumer goods supply chain consider demand volatility and forecast accuracy a high businesses priority, half still rely on Excel spreadsheets for forecasting.

Relying on Excel for forecasting is like relying on:


  • a Longship to get you across the Atlantic

  • your first guess on Let’s Make a Deal to be the right one

  • a shareholder proxy getting on the ballot at a Fortune 500

  • Florida surviving a hurricane season without any major city suffering damage

  • the price of fuel going down and staying down for an upcoming series of spot buys

  • natural resource supply to be consistent and predictable year-over-year

  • a flip of a fair coin to come up heads seven times in a row

Now, it’s true that:


  • the Vikings did make it across the Atlantic in a Longship, but a single storm could sink it

  • the first door you pick, with one-in-three odds, could be the right one, but the odds are actually twice as good if you switch

  • an activist shareholder can sometimes get a proxy on the ballot if he or she has enough time and money, but as pointed out by John Gillespie and David Zweig in Money for Nothing (How the Failure of Corporate Boards is Ruining American Business and Costing Us Trillions), examples are few and far between

  • even though no storms made landfall in Florida in 2011, this is Not a common occurrence

  • gas prices did consistently drop in the USA between September 2008 and December 2008, but have been otherwise steadily rising for the last five years

  • in some years the rice, sugar, and corn crops are almost the same as in the previous year, but given the increase in hurricanes, tsunamis, droughts, and other natural disasters in recent years, this is not a common occurrence

  • yes, heads can come up seven times in a row when flipping a fair coin, but the chances of this happening are less than 1%

In other words, you can forecast with Microsoft Excel, but your chances of doing well, especially given that 90% of spreadsheets have non-trivial errors (and collectively cost enterprises billions, as Fidelity and Fannie Mae found out), are (vanishingly) small (as the complexity of the forecast increases). One has to remember that there’s no intelligence behind a spreadsheet and they are just a source of peril that can cost your organization millions without anyone noticing.

Consumer Sustentation 74: Demand Planning

Demand Planning is a damnation. Why? As per our original damnation post,

  • traditional demand planning models require historical data
  • traditional demand planning models require market predictability
  • traditional demand planning models require market foresight
  • traditional demand planning requires knowledge of the expected price point

And how often in today’s constantly changing consumer marketplace, with new product releases coming faster and faster (to the point where your phone, laptop, and music device is out-of-date by a whole new release within a year), do you have good historical data, market predictability, and foresight? And how often can you be confident in the price-point, as a skunk-works product release by a competitor between sourcing and sale can force a price reduction to prevent inventory sitting on the shelves indefinitely.

So what can you do? (Besides burying your head in the sand like an ostrich?)

1. Get as much market data as you can.

Collect as much data as you can on your competitors imports, sales, and revenue using publicly accessible import data, analyst data, and company annual reports. It won’t be accurate, but with enough data you can often identify better trends than you could on the most similar product in your own inventory (which might not be similar, or recent, enough to be sufficiently relevant).

2. Have third parties conduct surveys on your behalf.

Sometimes the best way to gauge a market forecast is to actually conduct customer surveys and have a third party use the data to estimate demand for you. If you have no clue, the best thing you can do is admit it and get an expert to help you come up with a realistic demand forecast range.

3. Don’t focus a number, focus on a range and a potential rate of ramp-up or ramp-down.

If you know the demand is expected to be in the 100K to 200K units a month range, and the demand could double overnight, then you know that you need to contract for the low-end, but with a supplier that could ramp up to double production in a matter of weeks if necessary. And you have to negotiate a contract that allows orders to escalate, with pre-defined increases if the supplier is forced to work overtime (so you don’t get any billing surprises or animosity down the road).

4. Keep on top of sales data in real-time.

Be sure to get at least weekly PoS updates, and re-run the projections on a regular basis to detect an upswing or downswing early, so that you don’t get caught with your pants down, or, even worse, your pants off.

If you follow these tips, then you can get a reasonable grip on demand planning while your competitors flounder with the flounders.