Keeping a human in the loop deliberately, not decoratively

5 days ago
3 min read

Updated: 2 days ago

Introduction

"A human reviews the output" is the standard reassurance offered whenever automation touches something important, and it is frequently untrue in practice. A person clicking approve on forty items an hour is not reviewing them; they are providing a signature. The control exists on the process diagram and not in the work.

Designing a review that genuinely functions requires deciding what the person is checking, giving them the information and the time to check it, and making it acceptable for them to reject things. None of that happens by default, and a review that cannot realistically change the outcome is worse than no review because it creates the appearance of a control.

1. Keeping a human in the loop deliberately means the review can change the outcome

The test to apply.

How often does the reviewer reject or amend something? If the answer is almost never across hundreds of items, the review is not functioning, whatever the process document says. A healthy review rejects or amends a small but non-trivial proportion, and knowing what that proportion is for your process is worth measuring.

2. Say what the person is checking

Specificity makes review possible.

"Check the figures against the source" is actionable. "Review before sending" is not, and it produces a glance. A short explicit checklist is what converts an approval step into a control. Four items is usually enough, and the list should come from the errors you have actually seen.

3. Give them the information they need

Review requires context.

The output alone is not reviewable. The source document, the previous version, the customer history or the original request has to be available in the same place, or the reviewer is checking plausibility again.

4. Set a volume the person can actually handle

The arithmetic of attention.

If proper review takes two minutes and three hundred items arrive daily, the review is impossible and somebody will resolve that by not doing it. Either the volume routed for review reduces or the resource increases.

5. Route by risk rather than reviewing everything

The design that works.

High-value, unusual, customer-facing or low-confidence items get reviewed; routine ones pass. This concentrates the available attention where it changes something and makes the review sustainable. The thresholds should be written down and revisited, not left to whoever is on duty.

6. Make rejection normal and easy

Otherwise it does not happen.

If sending something back is slow, awkward or treated as obstruction, reviewers stop doing it. The rejection path should be as easy as the approval path, and its use should be treated as the control working.

7. Give the reviewer the authority they need

Responsibility without authority fails.

A junior person nominally reviewing output that a senior colleague generated will not challenge it. The review has to sit with someone who can say no without it being a difficult conversation.

8. Audit the review, not only the output

The check on the check.

Sample items that were approved and verify them independently. This is the only way to know whether the review is functioning, and it is almost never done.

9. Reconsider it periodically

Both directions.

Sometimes the review is no longer needed because the process has proved reliable. Sometimes it needs strengthening because the volume has grown. Neither is noticed without a deliberate review of the review.

Where decisions affect individuals — employment, credit, service refusal, pricing that differs between people — several jurisdictions impose specific requirements about human involvement and about a person's right to challenge a decision. A nominal review may not satisfy those, and the position should be confirmed.

Conclusion

Design a review that can realistically change the outcome, or acknowledge that you do not have one.

State explicitly what the reviewer is checking, give them the source material alongside the output, size the volume so proper review is possible, route by risk instead of reviewing everything equally, make rejection as easy as approval and treat it as the control working, place the review with someone who has the authority to say no, sample approved items to audit the review itself, and reconsider the arrangement periodically in both directions.