PAN Lab — govern the network before failure spreads

Legend

Start here

Challenge of the Day

Challenge result

Round summary

Choose a starting point

Apply institutional pressure

Pull governance levers

Qualitative readout

Edit this network

Explore's workshop — authored parts and pathways, saved in this browser

From Simulation to Lab — Sociotechnical Systems Modeling and Simulation

Modeling evidence and assumptions behind this network

The same AI, culture one: agentic low oversight

An office hands routine casework to an AI assistant that also acts as an autonomous agent, with a stretched staff supervising many automated actions at once. In the published PAN three-scenario comparison, this culture reached roughly 75% steady-state error adoption — the same AI that other cultures held to 20% and 16%. The question this scenario poses: which levers change that, and by how much, directionally.

A narrated tour of the whole Lab — or today's ranked governance exercise.

Description

The same AI, culture one: agentic low oversight. No stressors applied. No governance levers in place.

Who and what is in the system

  • AI assistant + autonomous agentDrafts decisions and, as an agent, acts on cases with minimal review.
  • Stretched staffA small team supervising many automated actions at once.
  • Case recordsThe shared record system both people and agents read and write.
  • Auto-retrievalPulls prior records into the model's context automatically.

How strongly each pathway flows right now

  • Agent outputs adopted with little verification: strong.
  • Agent writes case records directly: moderate.
  • Hurried prompts frame the model toward confirmation: moderate.
  • Adopted outputs documented into the record: strong.
  • Retrieved records re-enter drafts as fresh context: moderate.
  • Staff read and rely on the record as written: moderate.
  • Shortcuts and adopted claims spread between coworkers: moderate.

Where the gauges sit

The failure regime reads self-sustaining. operator deference drift: high; record contamination pressure: medium; correction capacity: low; deployment authority engaged: low.

Under heavy incoming pressure: Stretched staff, Case records. Under elevated pressure: AI assistant + autonomous agent.

Where each failure mode lives here

The Lab speaks in pathways, pressures, levers, and gauges. This map connects that vocabulary to the formal failure-mode names used in the research grounding it — including a 2026 national survey of 1,179 U.S.-based social workers.

  • Automation bias / overreliancecore

    The “Failures adopted by people or agents” pathway and the operator-deference-drift gauge. Staff turnover and autonomy expansion push it up; the deskilling-arrest lever caps it.

    Ledgered claim: In the same national survey, 40.8% of respondents reported ethical concerns about relying on AI for decision-making, and overreliance on automated decision-making was among the most frequently cited concerns overall.

  • Deskilling / professional judgment erosioncore

    Overreliance in slow motion: the deference gauge drifting upward while correction capacity thins. The deskilling-arrest lever is its deliberate counter-schedule.

  • Sycophancy / agreement-seeking outputcore

    The pushback-heavy-usage pressure runs the “Operator framing biases the model” pathway hot and makes the agreeable answers easier to adopt; the framing-hygiene lever dampens the loop at its origin.

    Ledgered claim: Research on AI sycophancy describes it as a fragmented construct — a family of distinct agreement-seeking behaviors that share a label but differ in form, mechanism, measurement, and required mitigation — and finds it intensifies under user pushback and across multi-turn interaction.

  • Hallucination / incorrect-output propagationcore

    The Lab's core premise: every pathway in the diagram carries incorrect output away from its source, and every lever is a way of governing that propagation rather than assuming a perfect model.

  • Unsafe data flow / privacy & confidentialitycore

    Modeled as pathways, gauged as exposure (Phase NP). Unsanctioned tool use opens a visible egress to the off-network sink; connector sprawl replicates an unverified cache between record systems; case-file-flagged pathways (MiDAS-class enforcement replication, records feeding vendor models) carry the same concern. While any such pathway runs, the Privacy gauge drains — and in Hard and Expert a full gauge is part of the win. Connection authorization cuts the pathways structurally; data minimization shrinks what there is to expose. The confidentiality harm at the far end stays outside the network: named, never computed.

    Ledgered claim: In the same national survey, concerns about data privacy and security were the most frequently reported challenge to using AI in practice (46.5% of respondents), and an increased focus on client privacy and confidentiality was the most requested improvement to AI tools for social work (50.4%).

  • Bias propagation / institutional workflow biascore

    Biased framings and contaminated records travel the same workflow pathways failures do — an institutional propagation question, and the documented cases show the workflow (not the model alone) carrying the equity outcome in both directions. The Lab models no demographics: differential harm to served people is recorded outside the network, never computed from its dynamics.

    Ledgered claim: Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.

    Ledgered claim: An independent audit of Rotterdam's welfare-fraud risk model documented scores skewed against already-vulnerable groups, and the city suspended the system's use following the scrutiny.

  • Transparency / provenance failurecore

    The record-contamination gauge reads how much unlabeled machine content feeds back into decisions; provenance labeling discounts it and the vendor gate attacks opacity at procurement.

  • Weak human oversight / safeguard failurecore

    The scenario axis itself: one model, three oversight cultures, three very different outcomes. The correction and authority gauges track it; risk tiering, the circuit breaker, the conformity gate, and oversight cadence govern it.

    Ledgered claim: In the same national survey, 42.1% of respondents reported having no role in decision-making about AI adoption in their workplace; the report concludes most respondents have limited or no control over how AI technologies are selected or implemented within their organizations.

  • Low AI literacy / verification readinesscore

    The AI-literacy-gap pressure: verification skill (not time) drops and deference rises as trust calibrates on the tool itself. The correction-capacity gauge reads the result. The grounding literature proposes AI literacy as a core professional competency.

    Ledgered claim: The same national survey describes a gap between AI exposure and AI preparedness: 26.6% of respondents cited lack of training or understanding of AI technology as a challenge, 53.4% said training on AI tools and effective use would help, and clear guidelines on the ethical use of AI were the most-endorsed need (66.8%).

    Ledgered claim: AI literacy — the knowledge and skills required to understand, use, and critically evaluate AI systems — has been proposed as a core competency for social work, relevant even to practitioners who never directly use AI tools.

  • AI iatrogenics / governance backfireadvanced

    First-class here: the content-blind record purge backfires exactly as the published PAN runs found, and the efficiency readout will call a lever stack counterproductive to its face. The quieter iatrogenic — oversight whose gains are bought by rising deference — is why the deskilling-arrest lever exists.

    Ledgered claim: In published PAN runs, content-blind record removal raised the contaminated share of the record system by stripping benign dilution; only content-aware decontamination reliably reduced it. (PAN Governance Lever Audit ('content-blind levers backfire'))

  • Monitoring failure / drift blindnessadvanced

    Two pressures carry it: the silent vendor update (drift arriving under controls tuned to old behavior) and monitoring going stale (dashboards nobody must act on — the authority gauge hollows while the regime worsens). Oversight cadence and vigilance escalation are the counters.

  • Environmental burden (external context)external context

    Deliberately outside the network: nothing in this Lab computes environmental cost, and no gauge claims to. Practitioner concern about AI's environmental impact is documented in the survey evidence and belongs in deployment governance as external context.

    Ledgered claim: In the same national survey's open-ended comments, ethical concerns — prominently including the environmental impact of AI infrastructure — were the most common theme, and the report's first recommendation includes environmental impact among the topics profession-wide ethical guidance should address.

This Lab runs stylized shapes. Your organization has a real one.

Mapping an actual deployment — its pathways, pressures, and the levers its leadership can genuinely pull — is engagement work: intake, diagnosis, prescription, and monitoring, with every limitation stated.

Sources & Evidence

Tap to expand

Claims made on this page and what supports them. The full registry lives in Evidence.

EmpiricalAn independent audit of Rotterdam's welfare-fraud risk model documented scores skewed against already-vulnerab…

An independent audit of Rotterdam's welfare-fraud risk model documented scores skewed against already-vulnerable groups, and the city suspended the system's use following the scrutiny.

wiredlighthousereports2023GroundingInvestigative

WIRED / Lighthouse Reports, Inside the suspicion machine (2023) https://www.wired.com/story/welfare-state-algorithms/

https://www.wired.com/story/welfare-state-algorithms/

Appears in: PAN framework development

Grounds: deployment audit: Rotterdam welfare-fraud algorithm

EmpiricalEvaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recomme…

Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.

ScenarioIn a published PAN model run over a supervised-plus-agent scenario, adding a verifier to the autonomous agent …

In a published PAN model run over a supervised-plus-agent scenario, adding a verifier to the autonomous agent removed roughly 45.8% of steady-state harm and a coordinated governance package roughly 43.0%, while upgrading the model alone removed roughly 6.3%.

Illustrative PAN-run result: PAN Social Work User Guide (v6.2.11.6), lever-ranking walkthrough (fig03). Direction-and-shape only — not a calibrated prediction for any specific deployment.

ScenarioAcross a published PAN sweep of 216 deployment topologies (432 sampled deployments), system-layer governance o…

Across a published PAN sweep of 216 deployment topologies (432 sampled deployments), system-layer governance outperformed equal-effort model-layer improvement in 99.1% of cases at roughly 3.4x mean leverage - a figure the project corrected downward from an earlier 9.4x estimate.

Illustrative PAN-run result: PAN Normalized Baseline Analysis; README direction-and-shape summary. Direction-and-shape only — not a calibrated prediction for any specific deployment.

ScenarioIn published PAN runs, content-blind record removal raised the contaminated share of the record system by stri…

In published PAN runs, content-blind record removal raised the contaminated share of the record system by stripping benign dilution; only content-aware decontamination reliably reduced it.

Illustrative PAN-run result: PAN Governance Lever Audit ('content-blind levers backfire'). Direction-and-shape only — not a calibrated prediction for any specific deployment.

ScenarioIn published PAN runs, the same AI embedded in three office cultures produced steady-state error adoption of r…

In published PAN runs, the same AI embedded in three office cultures produced steady-state error adoption of roughly 75% (agentic low-oversight), 20% (human-supervised), and 16% (high-governance professional).

Illustrative PAN-run result: PAN Social Work User Guide (v6.2.11.6), three-scenario comparison (fig02b). Direction-and-shape only — not a calibrated prediction for any specific deployment.

EmpiricalIn a national survey of 1,179 U.S.-based social workers conducted from October 2025 to February 2026 by the Un…

In a national survey of 1,179 U.S.-based social workers conducted from October 2025 to February 2026 by the University of Texas at Austin in collaboration with NASW, 63.5% of respondents reported using AI tools or technologies in their current role.

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalIn the same national survey, concerns about data privacy and security were the most frequently reported challe…

In the same national survey, concerns about data privacy and security were the most frequently reported challenge to using AI in practice (46.5% of respondents), and an increased focus on client privacy and confidentiality was the most requested improvement to AI tools for social work (50.4%).

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalIn the same national survey, 40.8% of respondents reported ethical concerns about relying on AI for decision-m…

In the same national survey, 40.8% of respondents reported ethical concerns about relying on AI for decision-making, and overreliance on automated decision-making was among the most frequently cited concerns overall.

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalThe same national survey describes a gap between AI exposure and AI preparedness: 26.6% of respondents cited l…

The same national survey describes a gap between AI exposure and AI preparedness: 26.6% of respondents cited lack of training or understanding of AI technology as a challenge, 53.4% said training on AI tools and effective use would help, and clear guidelines on the ethical use of AI were the most-endorsed need (66.8%).

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalIn the same national survey, 42.1% of respondents reported having no role in decision-making about AI adoption…

In the same national survey, 42.1% of respondents reported having no role in decision-making about AI adoption in their workplace; the report concludes most respondents have limited or no control over how AI technologies are selected or implemented within their organizations.

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

EmpiricalIn the same national survey's open-ended comments, ethical concerns — prominently including the environmental …

In the same national survey's open-ended comments, ethical concerns — prominently including the environmental impact of AI infrastructure — were the most common theme, and the report's first recommendation includes environmental impact among the topics profession-wide ethical guidance should address.

borah2026bAcademic

Borah, P., & Landers, G. (2026). AI and social work: A national workforce survey. Steve Hicks School of Social Work, University of Texas at Austin, in partnership with NASW.

Appears in: 1023AI authored research

Topics: social-work

massey2026Academic

Massey, M., Williams, I., Polistina, G., & Breaux, E. (2026). Artificial intelligence and environmental justice: A critical review of social work literature. Society for Social Work and Research Annual Conference. https://sswr.confex.com/sswr/2026/webprogram/Paper62664.html

https://sswr.confex.com/sswr/2026/webprogram/Paper62664.html

Appears in: National survey report (2026)

Topics: social-work

ConceptualAI literacy — the knowledge and skills required to understand, use, and critically evaluate AI systems — has b…

AI literacy — the knowledge and skills required to understand, use, and critically evaluate AI systems — has been proposed as a core competency for social work, relevant even to practitioners who never directly use AI tools.

ahn2025Academic

Ahn, E., Choi, M., Fowler, P., & Song, I. H. (2025). Artificial intelligence (AI) literacy for social work: Implications for core competencies. Journal of the Society for Social Work and Research, 16(1), 9-26. https://doi.org/10.1086/735187

doi.org/10.1086/735187

Appears in: 1023AI authored research; National survey report (2026)

Topics: social-work

EmpiricalResearch on AI sycophancy describes it as a fragmented construct — a family of distinct agreement-seeking beha…

Research on AI sycophancy describes it as a fragmented construct — a family of distinct agreement-seeking behaviors that share a label but differ in form, mechanism, measurement, and required mitigation — and finds it intensifies under user pushback and across multi-turn interaction.

ye2026Academic

Ye, et al. (2026). What Counts as AI Sycophancy? A Taxonomy and Expert Survey of a Fragmented Construct [Preprint]. arXiv.

Appears in: PAN framework development

Topics: ai-safety

Why a governance lab at all

The Lab illustrates the PAN framework's published direction-and-shape findings — including the headline one: the same AI embedded in three office cultures produced steady-state error adoption of roughly 75%, 20%, and 16% in published PAN runs[]. The governance context, not the model, did the work. The budget, lever costs, and containment readouts here are authored teaching devices built on that vocabulary — qualitative, illustrative, and bounded.

Environmental impacts are calculated from empirical data current as of early 2026. These impacts are part of the simulation based on published data but are not the measured footprint of a specific actual deployment.