10 to the 23 AI logo

Domain Atlas / Child welfare & family services

Case fileAllegheny County, Pennsylvania, USAlarge deployment

Allegheny Family Screening Tool

Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.[3]

What happened

Allegheny County's Family Screening Tool scores child-maltreatment referrals to support call-screening decisions, and is unusual for the depth of its public documentation: methodology reports, an independent impact evaluation, and sustained academic and journalistic scrutiny. That record includes findings on racial disparity in screen-in rates — and, notably, evaluation evidence that screener overrides of the tool's recommendations reduced the disparity relative to the tool alone, alongside long-running community and researcher criticism of using such tools at all.

The sociotechnical reading

AFST reframes "human in the loop" from a checkbox into a measurable component: here, the operator network demonstrably changed the system's equity behavior, in the protective direction. That cuts both ways — a system whose fairness depends on engaged overrides inherits every fragility of the humans doing the overriding (workload, deference drift, deskilling). It is the Atlas's clearest bridge to the deskilling and vigilance material in the Field Guide: the safeguard is alive, so it must be maintained like something alive.

The concepts used in this reading are defined in the Field Guide; the governance responses live in the Practice Library. A stylized system in this case's shape can be stress-tested in the PAN Lab.

Grounding sources for this case

The same sources that ground this model organization in the PAN library — evaluations, government documents, investigative reporting, and advocacy documentation, each labeled by tier.

stapletonetal2022GroundingPeer-reviewed

Stapleton et al., Imagining new futures beyond predictive systems in child welfare (FAccT 2022) https://dl.acm.org/doi/10.1145/3531146.3533177

https://dl.acm.org/doi/10.1145/3531146.3533177

Appears in: PAN framework development

Grounds: deployment audit: Allegheny AFST

Topics: child-welfare

Seeing your organization in this case file?

The histories here are documented after the harm. Mapping a live deployment's pathways and pressures — before the incident report — is engagement work: intake, diagnosis, prescription, and monitoring, with every limitation stated.

Sources & Evidence

Claims made on this page and what supports them. The full registry lives in Evidence.

EmpiricalEvaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recomme…

Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.