Benefits navigation & public-facing chat
Conversational systems answering high-stakes questions for the public — where an authoritative wrong answer is indistinguishable, to its victim, from policy.
What AI is doing here
Benefits-navigation chatbots
GenerativeConversational guidance about eligibility and process, for applicants directly or for navigators assisting them.
Retrieval-grounded appeal drafting
GenerativeRAG systems drafting determinations or appeal responses from policy corpora for adjudicator review.
What has gone wrong — and right
Documented deployments, presented as stylized model organizations with full citations.
Who is in the system
- Frontline workers. Caseworkers, screeners, eligibility staff — the operator network whose judgment the system augments or erodes.
- Agency leadership. Owns procurement, policy, and the authority map; answers for the system publicly.
- Served people & families. Those the decisions land on. Deliberately outside the PAN dynamics — their outcomes are measured, never simulated.
- Vendors. Build and update the systems; hold the information asymmetry procurement must govern.
- Advocates & community organizations. Surface harms institutions do not see; historically the earliest accurate signal.
Dominant pressures
- Caseload surge. Demand outruns staffing; per-case attention shrinks and review becomes triage.
- Vendor opacity. The deploying institution cannot inspect the model, data, or update pipeline it is accountable for.
- Data & policy drift. The world, the intake process, and the rules change under a system trained on how things used to be.
Questions leaders should be asking
- 1. Is the system answering people directly, or assisting a professional who answers?
- 2. What is the retrieval corpus, who curates it, and how fast does it track policy change?
- 3. How are confident wrong answers detected — before a journalist detects them?
- 4. Does the interface honestly convey uncertainty, or does it perform authority?
For the actions behind these questions, see the Practice Library.