Ch. 16 — Human-in-the-Loop — When Agents Must Stop and Ask

Human oversight is not a UX preference in agentic systems. It is a compliance requirement that must be engineered in, not bolted on.

An agent that processes a loan application, generates a clinical recommendation, or terminates an employment contract is making a consequential automated decision. The frameworks are explicit about what that requires:

These are not preferences. They are architectural constraints. If a human cannot see what the agent is doing, intervene before or after the fact, and contest its decisions, the system is non-compliant by design.