01
Related
Where to read next on the same subject.
02
Short answer
Show an operator four numbers: WAPE for error in units they recognize, bias for the high-or-low tendency, skill against the seasonal baseline, and the hit rate on the decision band that matters. Cut all four by segment. A single averaged accuracy number is where local failure goes to hide.
A supply-chain head once told me her forecast was "ninety-two percent accurate," and she was right, and her stores were still angry. The number was a weighted average across a portfolio that included steady sellers, promotions, and a long tail of intermittent items. The steady sellers were near-perfect and carried the average. The intermittent tail, where the stockouts lived, was wrong in both directions, and no report had ever shown it to her, because the report showed one number and the number was fine.
Accuracy work consists largely of refusing to hide behind that one number.
03
The four numbers
WAPE, the weighted absolute percentage error, divides total absolute error by total volume, which gives the error in units planners can feel: "we missed by about three days of sales." MAPE, the per-item mean, explodes whenever any item's volume nears zero, which is exactly where interesting failures live. Report WAPE as the headline. Keep MAPE in the drawer for comparing with vendors who insist on it, and open the drawer rarely.
Bias is the tendency to run high or low over time, and it is the number with teeth. A forecast with modest error and persistent low bias quietly starves shelves while looking acceptable on the accuracy report. Bias is also the cheapest error to correct: a correction rule per region or category, reviewed monthly, recovers more service than most model upgrades. Track it by period and by segment, and treat a persistent sign as a finding with an owner, never as noise.
Skill is the gain over the baseline, and the baseline is the seasonal naive: what happened last year, or last week, shifted appropriately. A complex method earns its running cost by beating that line, per segment, per period, and the comparison happens on the same cutoffs every time. A model that shows impressive accuracy and no skill is a confession wearing a dashboard: it is more expensive than last year, at the same quality.
Hit rate closes the set with the decision in view. The forecast exists to trigger orders, staffing, or preparation inside a band, so count how often it lands inside the band where the decision would be made. A forecast can be mediocre by WAPE and excellent by hit rate, or the reverse, and the operating team should know which quality they are buying.
04
Cut by segment, always
Averages hide the segments that fail, and the failing segments are where the stockouts, the waste, and the angry calls live. Review every metric by region, category, and horizon. Give intermittent items their own view, with their own realistic targets, because a fast-moving item's accuracy standard is meaningless for a part that sells twice a month.
Flag promoted periods rather than deleting them, because the promotion behavior is real signal about the demand the business creates, and deleting it flatters the model while unteaching it. The flag exists so both readings are available: the error with the promotion, and the error without it.
The segment table should look like the business. One page, the planner's own units, worst ten items listed by name. When the table needs a statistician to interpret, it will be interpreted by nobody, and the next surprise will be filed under "the forecast was wrong again."
05
Against a baseline, in writing
Name the baseline in the evaluation document, with its cutoff rules, and never change it mid-quarter to flatter a result. The comparison discipline is what makes skill meaningful: same periods, same segments, same treatment of promotions and holidays. The BluePi patent work on retail demand forecasting started from exactly this discipline, because a forecasting method claim that cannot name its baseline and cutoffs is a claim about presentation rather than performance.
06
The monthly review, run by operators
What gets shown: last month's WAPE and bias per region, skill against baseline, worst ten items, and the overrides overlayed on the misses, with the planners' reasons. What gets decided: a biased region gets a correction rule, a failing segment drops to the fallback method, thresholds update for the season, and every decision lands in the record with an owner and a date.
The value of that meeting is the correction, fallback, or threshold decision it produces. A forecast program is judged by whether its review changes operating behavior, and behavior changes only when the numbers shown are numbers operators can act on. Precision that cannot reach a decision is decoration.
07
Two pitfalls worth naming
MAPE near zero-volume items is the classic, and the fix is structural: report WAPE as headline, segment the tail, and never let a portfolio average near a review. The second pitfall is symmetric scoring in a business with asymmetric costs: in most operations, an under-stock hurts more than an over-stock, and an evaluation that treats the two directions equally will select models that are pleasant to grade and expensive to run. Weight the errors the way the business feels them, and say so in the evaluation document.
08
A worked example
Numbers make the metrics concrete, so take one region, one month, one category. Total actual sales were 100,000 units. The forecast totaled 108,000. Absolute errors by week came to 9,000 units in total, so WAPE is 9,000 over 100,000, or 9 percent. Bias is plus 8,000 units over actual, a persistent over-forecast of 8 percent, which is the finding: modest error, wrong direction, four consecutive weeks. The correction rule for this region writes itself, and it is worth more service than another month of model tuning.
Skill needs the baseline on the same weeks. Seasonal naive, the same weeks last year, misses by 11,000 units. The model misses by 9,000. Skill is 18 percent, which sounds decent until the segment table shows where it came from: the promoted items contribute nearly all of it, and the non-promoted remainder shows skill of 2 percent, inside the noise. The honest reading is that the model earns its cost on promotions and is barely ahead of last year elsewhere, which points the next effort at the non-promoted items, not the promotions.
Hit rate uses the decision band. Orders trigger when forecast demand crosses a case threshold, and the question is how often the forecast put the week on the correct side of that line. It did so for 34 of 40 store-weeks, and the six misses are all low-volume stores where a two-case error flips the decision. Those six stores are the tail that a segment view would have surfaced a quarter ago.
The monthly table assembles the same arithmetic for every region and category, with the worst ten items named. The discussion takes the numbers for granted and spends its time on the decisions: which correction rules to set, which segments drop to the fallback, and where planners' overrides beat model output. The arithmetic is deliberately simple. The discipline of cutting it by segment and against a baseline is what carries the value.
09
Related
The methods guide covers how candidate models are chosen and compared. The factors guide lists what shapes the method for a given decision. The patent page describes the filed work behind our retail forecasting practice, and the QSR case study shows hourly forecasting measured against a live operating decision.