01
Short answer
Put humans where the cost of a wrong answer exceeds the cost of waiting, and automate everything around that decision: routing, prioritization, review interfaces, and the feedback that flows back into your eval dataset. The failure mode is not having human review; most systems have some. The failure mode is undifferentiated review, humans checking a random sample of everything, missing the cases that matter, and burning out, while the automation never learns because the review output goes nowhere. Use review to contain immediate risk and, where permitted, improve the evaluation dataset. Record the outcome of each review. A review can be valuable even when it does not become training data.
02
Decide where the human belongs
The placement question is economic, not technical. Estimate the cost of a wrong automated answer for each output type, wasted user time, a wrong invoice, a compliance breach, against the cost of human latency for that same output. Where the error is cheap and reversible, automate fully and let feedback buttons handle the residue. For outputs with serious or irreversible consequences, define the required human authority and review gate before release. Specify which decisions the automation may recommend and which a qualified person must authorize. The middle band, whose size depends on the workflow, is where judgment calls live, and this band is where most designs get lazy. They either over-automate it and absorb the error rate, or over-review it and absorb the labor cost.
Confidence routing is the usual mechanism for the middle band: outputs the system scores as high-confidence go straight through, low-confidence ones queue for review. Two cautions. Confidence scores from LLMs are poorly calibrated out of the box, a model can be confidently wrong, so validate the threshold against your own error data before trusting it. And keep the routing logic versioned and monitored like any other release artifact, because a silent change to the threshold quietly moves work between machines and humans, and the bill shows up in two different places.
03
Build the review interface for decisions, not reading
Reviewers should not read full conversations to find the problem. Show them the model output, the retrieved context it was grounded in, the confidence score, and highlights of claims that failed a check, with the reason for the flag. Give them one-click verdicts, correct, wrong, needs edit, and make "wrong" capture a category, not just a flag, because an uncategorized wrong is an unusable training signal. Measure reviewer throughput and agreement. If two reviewers regularly disagree on the same output type, the task definition is ambiguous, and no amount of review labor fixes that; fix the rubric first.
Watch for review drift over time. Reviewers who see mostly correct outputs start rubber-stamping; reviewers who see mostly wrong ones start rejecting everything. Use blinded spot checks, consistent rubrics, and appropriate reviewer rotation. Compare agreement and error rates on comparable case types; approval-rate differences can reflect different case assignments.
04
Close the loop or don't bother staffing it
Every review verdict should flow somewhere concrete. Confirmed-wrong outputs with categories go into the eval dataset as new test cases. Reviewed corrections can become prompt tests or training candidates after permission, data-quality, and privacy checks. Keep the held-out evaluation set separate from training data. Category patterns aggregate into weekly reports that tell you which part of the system to fix next. If review output only fills a dashboard nobody reads, you are paying human wages for a metric that changes nothing. The strongest version of this loop compounds: review volume shrinks as the system improves, because each fix removes a class of errors from the queue. A growing queue can reflect higher traffic, different case difficulty, stricter routing, slower handling, or declining model quality. Track arrival rate, review share, handling time, queue age, and repeat failures before deciding what to change.
05
Staff it like operations, not like a project
Human review is an ongoing operational function with staffing, quality management, and tooling costs, and it should be budgeted and owned that way from the start. Pilot systems hide this because a founder reviews outputs after dinner and calls it human oversight. As traffic grows, informal review can exceed the time available to the reviewer. Decide early what the review SLA is, how long an item can sit in the queue before the user experience breaks, because the SLA drives staffing math. An eight-hour review service-level agreement (SLA) and a two-minute one require different capacity. Estimate daily review hours from arrival volume, review share, and average handling time. Allow capacity for peaks, absences, quality checks, and escalations. Define the action when the queue misses its deadline.
06
Define a decision that a reviewer can own
Consider a support assistant that recommends a refund. This is an illustrative workflow. The reviewer needs the proposed action, the applicable policy, the transaction details they are authorized to inspect, and the checks that triggered review. A score alone does not explain whether the recommendation is permitted. The interface should make the relevant rule and missing information visible.
Separate reviewing an answer from authorizing an action. A person may agree that an explanation is accurate while lacking authority to approve a payment. Record the decision type, permitted actor, and authorization boundary. Where the workflow needs two approvals, the application must enforce that requirement rather than display a reminder and continue.
Capture a reason code and any correction alongside the verdict. Use a short list of categories that supports actual engineering decisions, such as missing source, wrong policy version, unsupported claim, or missing user detail. Provide a way to flag an unclear rubric. Forcing an ambiguous case into an inaccurate category creates misleading feedback.
07
Test routing against the cases it misses
A routing threshold determines which cases receive attention. Evaluate the errors that pass through automatically, not only the flagged cases. Sample from the automated path and obtain independent human labels. This reveals whether a reassuring score is hiding failures that the review queue never sees.
Measure unnecessary review as well. When correct low-risk outputs repeatedly reach humans, the team loses capacity for cases where intervention matters. Compare routing candidates using the cost of missed errors, the review load, and the delay introduced to users. Establish separate rules for workflows with different consequences.
Version the routing configuration and retain the reason for each change. A threshold adjustment can change staffing needs immediately. Before rollout, estimate the new arrival rate and check that the team can meet the review deadline. Keep a rollback option for a routing change that overwhelms the queue or misses important failures.
08
Manage queue overload explicitly
Define what happens when the oldest case approaches its deadline. The response may add qualified reviewers, limit intake, change user expectations, or hold the action. Do not silently remove a required approval to keep throughput high. A safe delay with a clear explanation is preferable to an unauthorized action.
Prioritize using consequence and time sensitivity rather than arrival order alone where the workflow requires it. Record the priority rule so reviewers can understand why a case moved ahead. Preserve an escalation route for cases outside the reviewer's authority or expertise.
09
Keep feedback useful and controlled
Review outcomes need a curation step before they become evaluation or training material. Confirm the correction, resolve conflicting labels, remove information that the next use does not permit, and record the source of the example. A rushed reviewer edit can contain its own mistake.
Track whether recurring failure categories decline after a fix, using comparable traffic. Reviewers should see the result of their feedback so they can identify a fix that did not work. Maintain separate records for operational authorization and model improvement: the first explains who permitted an action, while the second supports future changes to the application.
10
Related
Where to read next on the same subject.