Emergent misalignment

Alignment is usually verified as a snapshot: at this capability level, on these tasks, under these conditions, the system does what we intend. Emergence is what happens to snapshots. Scale the model, connect it to new data, hand it to a thousand new users with their own workflows, and the aggregate behavior of the system can change qualitatively — without any single component being at fault.

In institutional deployments the mechanism is often mundane. A tool adopted for drafting becomes, under deadline pressure, a tool for deciding. A recommendation that policy says is advisory becomes, in practice, determinative because overriding it requires a justification memo and nobody has time. Each step is locally reasonable; the sum is a system doing something nobody signed off on.

The hard part is that emergent misalignment is invisible to component-level checks. The model still passes its evaluations. Every individual is following an approximation of the rules. Only a system-level view — what is actually being decided, by whom, on what basis — reveals the drift.

That is the case for the anticipatory posture this site calls emergent foresight: governing for what the system is becoming, not just what it was at sign-off. The monitoring and circuit-breaker patterns in the Practice Library are its concrete tools.