01
Related
Where to read next on the same subject.
02
Short answer
A first deployment needs three numbers agreed in writing before build starts: a measured baseline of what the current process costs, a target tied to it with a period, and acceptance rules that say who signs and against what data. If any of the three is missing, the project will be judged on memory and opinion, and no system survives that review.
The worst review I have attended had no villain. A team had built a good document-extraction system, delivered on time, and demonstrated it end to end. The sponsor asked the only question that mattered: how much of the manual effort did it remove? Nobody could answer, because the manual effort had never been measured. Sixty seconds of agreement turned into forty minutes of recollection, and the project walked out with a "promising" instead of a "proven." The system was fine. The arithmetic was missing.
03
Measure the baseline alongside the operating team
A baseline is a measurement, so it needs a period, a source, and a witness. Take a real month, not a good one and not a holiday-skewed one. Count in the operating team's own units: hours per day spent hand-matching, error rate at month close, days of delay between event and decision. Watch the process once alongside its operators, because the documented process and the actual process are different documents, and only one of them is the baseline.
The measurement has one more property: both sides have to accept it. The delivery team can measure alone and produce a number the operating team will dispute at the review. Measure together, even if it takes a full day of someone's time. That day buys the only currency a target needs, which is agreement.
Record the baseline where decisions live, in the shared document with the scope and the risks. A baseline in a slide deck ages into folklore. A baseline in the record, with its period and method, can be quoted at every review for the rest of the engagement.
04
Set the target against the baseline
The target answers: how much better, by when, for whom. Tie it to the baseline explicitly. Hand-matching falls from six hours a day to under one by day 75. The exception rate drops below four percent for two consecutive months. The forecast review meeting starts thirty minutes later because the numbers arrive earlier. Each of these is countable, bounded in time, and visible to the person who owns the process.
Targets also need a secondary measure, because primary measures can be gamed accidentally. A team driving matching hours to zero can do it by rejecting more documents, which shifts cost downstream. The secondary measure, in that case, is the exception rate or the rework rate, agreed in the same session. Two numbers, chosen so that gaming one moves the other, is the smallest honest dashboard.
Someone has to own the target, and it should be the operating team rather than the delivery team. Ownership there means the reviews measure the change in their work, not the performance of a vendor. It also means they will defend the number when it is attacked, which happens at least once per engagement.
05
Agree acceptance before the first sprint
Acceptance rules convert a target into a decision. Which measures gate the cutover, who signs, against what data, and what happens if the numbers land short. Writing these rules down while everyone is optimistic costs an hour. Writing them during the review costs the engagement its momentum, because every rule will be negotiated under pressure.
Include the shadow period in the rules. State its length, what runs beside what, and how differences between the old and new process get explained. Include the rollback rule too: if the target slips, what does the team do, and who decides. Teams that rehearse the rollback rarely use it, for the same reason teams that rehearse fire drills rarely burn.
06
The three anti-patterns
The missing baseline is the quietest. "We do a lot of manual work" is not a number, and the project pays for the absence at the review, in front of the people who control the next budget. The fix costs a day of measurement and a week of scheduling.
The moving target is the loudest. Scope grows after the target is set, and the target quietly stays where it was, so the team chases a number that now describes a bigger system. Targets change, and they should when the world changes. But a target change is a written decision with a reason and a date, made in a review. A silent drift is neither.
The vanity metric is the most seductive. Accuracy on a clean test set is a fine engineering number and a poor business one, because it measures the model on the data the model was built for. The operating result lives in the workflow: what the team does with the output, and what it stops doing because of it. Measure that, or the test number and the review number will never meet.
07
What good looks like at day 90
The review takes fifteen minutes. The baseline is on screen with its period. The target is next to it. The measured result sits in the third column, with the data source named. Someone from the operating team confirms the number matches their experience. The sign-off happens in the meeting, and the handover conversation starts while the room still remembers why the system deserved to win. The three numbers exist for exactly that quarter hour.
08
When the baseline is disputed
Disagreement about the baseline is normal, and it has a cheap resolution: measure it together, in one day, on the floor. The delivery team and the operating team sit with the process for a shift, count the same work, and reconcile the two tallies in the evening. The number that emerges is smaller than either side's estimate, usually, because watching the work beats remembering the work.
The dispute often hides a definitional gap rather than a factual one. One side counts the rework that follows a bad hand-match; the other counts only the first pass. One side counts the analyst hours; the other counts only the clerical ones. The measurement session forces the definitions onto the table, and the agreed definition is itself a finding worth recording, because it becomes the acceptance measure's definition too.
When a full day on the floor is impossible, sample. Pick three representative days, instrument them lightly, and extrapolate with the error bars visible. The baseline does not need to be perfect. It needs to be agreed, bounded in time, and honest about its method, because the review will ask how it was measured and the answer "we watched" is unimpeachable in a way that "we estimated" never is.
The disputed baseline that cannot be resolved by either method is a red flag about the engagement rather than the number. If two teams cannot agree on what the current process costs, the operating model has a transparency problem that no system will fix, and the deployment should include fixing the visibility as part of its scope. Say that in the room, gently, and the rest of the engagement gets easier.
09
Related
The selection guide explains how the winning workflow earns its slot. The evaluation guide shows how the shadow period is scored. The hourly forecasting case study carries a measured baseline and result from live restaurant operations.