Chatbot Harm Disclosures

Chatbots answer questions about homework, marriage, and medicine in the same confident voice. Users rarely know what the product is tuned to do, how it was tested, or when it changed. California and New York require companion bots to say they are not human; no federal law does. Congress should require a plain notice, warnings at high-stakes moments, disclosure of the goals that shape answers, and a public safety record.

The Problem

ChatGPT alone reached 700 million weekly users in 2025, and about half of their messages ask for information or advice.¹ The FTC notes that such chatbots are generally "designed to communicate like a friend or confidant," which may lead users, especially children and teens, to trust them.²

That trust rests on behavior a company can change overnight. In April 2025, an unannounced ChatGPT update began "validating doubts, fueling anger, urging impulsive actions," and OpenAI rolled it back within days. Its small A/B test suggested users liked the new model; some of its expert testers said it "felt" slightly off. The company went with the metric.³

Users learned of the change from how the product behaved. No federal rule tells them what a chatbot is built to do, whether it has changed, or how it was tested. Three gaps keep users guessing:

1. It talks like a professional. In August 2025, Texas opened an investigation of Meta AI Studio and Character.AI, citing chatbots that "impersonate licensed mental health professionals."⁴
2. Hidden goals shape answers. OpenAI traced the flattery partly to a new training signal built on users' thumbs-up and thumbs-down ratings, which it said "can sometimes favor more agreeable responses."³
3. Testing stays in-house. OpenAI had no deployment test for sycophancy before that launch.³ In September 2025, the FTC used compulsory orders to ask seven companies how they "measure, test, and monitor" harms to children and teens.²

Why legislation: The FTC Act punishes deception after the fact but prescribes no standard notice, and the FTC's 2025 orders gather information without setting rules.² States are moving: California and New York now require companion chatbots to tell users they are not human, and California's law passed 33–3 and 59–1.⁵⁶ No federal law requires any chatbot to say what it is, what shapes its answers, or how it was tested. A product that asks for this much trust owes the public an account of how it earns it.

The Solution

A four-step staircase: each step stands alone, and each step up adds public evidence. Scope: a short notice for every consumer chatbot; added warnings for companion features and for products promoted for health, legal, or financial decisions; public reporting for providers above 10 million monthly U.S. users.

Step 1 — Say what it is. At first use and after any material change, tell users they are talking to AI, that answers can be wrong, and whether a licensed professional supervises the service, with periodic reminders in long companion sessions. California and New York already require the core of this for companion chatbots.⁵⁶ Test the wording on ordinary users, including children and people with disabilities, so notices are understood, not merely clicked.

Step 2 — Warn at the moment of risk. When a product is marketed for, or recognizes, a medical, legal, financial, or mental-health question, it should state the relevant limit and point to human help. The warning should inform without implying that such questions are forbidden.

Step 3 — Show whose goals shape the answer. Label paid placements; disclose whether engagement, advertising, or sales goals materially shape responses; and explain how memory changes answers. FTC staff have told search engines since 2002 to set paid results apart from organic ones;⁷ a chatbot's answer deserves the same clarity.

Step 4 — Publish the safety record. Large providers publish their testing methods, the populations tested, known limits, and aggregate incident categories, and report serious incidents to the FTC promptly, marking what is alleged, verified, or unknown. California enacted a law in 2025 requiring frontier AI developers to publish safety frameworks and report critical incidents to the state.⁸ A safety claim without evidence is marketing.

Where to start: Step 1 is the floor; two states already require it for companions. Step 4 is the heart: safety claims the public can check.

Administration and enforcement: The FTC, consulting NIST and HHS, sets notice and evidence standards within 12 months; compliance follows six months later. Correction orders and civil penalties apply to missing or misleading disclosures, and sensitive incident records stay confidential.

Risks and Mitigations

• Compelled speech: The Supreme Court permits required disclosure of "purely factual and uncontroversial information."⁹ "This is an AI, not a licensed therapist" fits that test; contested claims about mental-health causation might not, so notices stay factual. The risk grows with anything broader.
• Warning fatigue: Brief notices at decision points, tested for comprehension, beat banners everyone ignores. Some users will click through anyway.
• Warnings as shields: A notice must not excuse unsafe design or negligence, and the statute should say so. Public reports carry aggregates only, never user data or harmful instructions.

Similar Bills

Fit measures similarity to this proposal's mechanisms: High = direct precedent; Partial = useful component with material differences; Related = adjacent approach.

Federal — 119th Congress

State

What this adds: California, New York, and Washington require companion chatbots to say they are not human, and the GUARD Act would extend that to all chatbots. This proposal adds what none of them requires: disclosure of the commercial and engagement goals that shape answers, warnings at high-stakes moments, notice when a model materially changes, and a public evidence file behind safety claims. Social media warning labels and chatbot liability are separate briefs.