Field Guide / Why safety breaks under pressure
Deskilling and automation bias
Automation bias is acting on the system's output when you shouldn't (commission) or skipping the check you would once have done (omission). Deskilling is its slow-motion form: as reliance grows, the human capacity that oversight depends on erodes — often invisibly, because the paperwork still shows a human in the loop.
"A human reviews every recommendation" is the most common safeguard in institutional AI — and the most quietly perishable. Review is only a control if the reviewer engages: questions the output, brings independent knowledge, and is genuinely free to disagree. Each of those conditions erodes under load.
The erosion has two speeds. Fast: under deadline pressure, review becomes approval, because approving is one click and disagreeing is a justification memo. Slow: over months, the skills that made disagreement possible atrophy from disuse. The org chart still shows human oversight; the system behaves as if there is none.
What makes this dangerous is that it is invisible in output metrics while it is happening. Decisions keep flowing; agreement rates look like harmony. The tell is structural — rising deference, falling override rates, vanishing independent checks — which is why it must be watched for deliberately rather than noticed incidentally.
Countermeasures exist and appear in the Practice Library: deliberate deskilling-arrest steps (recertification, blind re-checks, rotation through unassisted work), override-rate monitoring, and honest workload design that gives review the time it actually takes. The PAN Lab's deference dynamics let you watch drift, and its arrest lever shows what deliberately freezing it does.
This page is conceptual framing — a way of seeing, not an empirical claim. Documented real-world events appear in the Domain Atlas with citations; testable versions of these ideas live in the PAN Lab.