NEDA: the helpline bot that gave dieting advice
The National Eating Disorders Association replaced its human helpline with a wellness chatbot. Within weeks an advocate posted screenshots of it recommending calorie deficits of 500 to 1,000 a day, weekly weigh-ins, and skin calipers, to the exact population the organization exists to protect. The bot came down in under 24 hours. The vendor had added generative AI to it without the nonprofit's knowledge, which is its own receipt about vendor change control.
What the verified record says.
- Organizations
- National Eating Disorders Association; Cass (chatbot vendor)
- When
- May 2023
- Failure class
- Silent wrong output silent-wrong-output
- Discovered by
- An advocate or consumer group advocate Not caught by the organization running the AI
- Missing control
- Output validation. Rule-based checks that parse what the model produced before a customer sees it. Full definition and the other incidents in this group → Watch this kind of control catch a planted failure → A recorded run of a certificate gate refusing a bad document →
- What would have caught it
- Rule-based limits on what a helpline bot may say to the population it serves, plus vendor change control so generative AI cannot be added without the operator knowing.
- Sources
-
- NPR primary
- Verification
- Confirmed: the claims held as first researched. The vendor had added generative AI to the chatbot without the nonprofit's knowledge, according to its chief executive. The advocate who surfaced it was a consultant rather than a psychologist, and a psychologist replicated the results before the bot came down. Verified against the primary source on August 12, 2026. Published on this site August 13, 2026, updated September 1, 2026.
This record is one of 26 in The receipts, each checked against a primary source before it is published. How the list is built →