Field Guide / Why safety breaks under pressure
AI iatrogenics
Iatrogenics is medicine's word for harm caused by the treatment. AI governance has the same phenomenon: controls, purges, filters, and review requirements that — deployed without regard to the system around them — create new risks even as they address old ones.
Every intervention has a footprint. A mandatory review step slows a queue, and a slowed queue creates pressure to rubber-stamp. An aggressive record purge deletes benign history along with bad, and can leave the surviving record set more contaminated, not less. A stricter filter pushes users toward workarounds that bypass governance entirely. The intervention did what it said — and the system did something else.
The point is not that safeguards are bad. It is that safeguards are interventions in a complex adaptive system, and interventions have side effects that component-level thinking never sees. A governance program that cannot name the side effects of its own controls is running blind in one eye.
Practically, this argues for three habits. Price the intervention: what load does it place on people, queues, and records, and who absorbs that load? Watch for displacement: when a pathway is blocked, where does the flow go instead? And prefer reversible moves when uncertainty is high, so that a control which backfires can be withdrawn without a crisis.
Every pattern in the Practice Library carries a "what can backfire" note for this reason, and the PAN Lab includes a deliberately instructive backfire — a content-blind record purge that makes things worse — so the phenomenon can be seen, not just described.
This page is conceptual framing — a way of seeing, not an empirical claim. Documented real-world events appear in the Domain Atlas with citations; testable versions of these ideas live in the PAN Lab.