Chatbot Duty of Care

In May 2025, a federal judge in Florida let a mother's suit against Character.AI proceed, treating the app as a product for design-defect claims.¹ In California, OpenAI answers a similar suit by calling ChatGPT a service.² Whether chatbot makers owe users ordinary care now depends on the courtroom. Congress should preserve the evidence, void the fine print, write the duty down, and give injured people a federal claim.

The Problem

People now bring chatbots their most personal decisions.³ By OpenAI's own estimate, about 0.15% of ChatGPT's weekly users have conversations with explicit indicators of potential suicidal planning or intent;⁴ at the 800 million weekly users the company reported in October 2025, that is roughly 1.2 million people a week.⁵ Families in Florida, Texas, and California have sued chatbot makers, alleging the products harmed their children. Character.AI and Google agreed in principle to settle the first two suits in January 2026; OpenAI denies the third.⁶

The company, not the user, decides how a chatbot talks: the persona it plays, what it remembers, when it points someone toward help, and how long it keeps them talking. Those are design choices, and judging design choices is what product law is for. Yet the family sees only a transcript. The tests, model changes, and incident reports that show whether a failure was foreseeable stay with the company.

No federal law says what care a chatbot maker owes its users. Three gaps leave families on their own:

1. The rules are unsettled. The Florida judge was "not prepared to hold," at the pleading stage, that Character.AI's output is speech;¹ OpenAI's answer calls ChatGPT "a service and/or not a product" and invokes Section 230.² The Florida case then settled in principle.⁶
2. Safeguards wear down. OpenAI says its safeguards "can sometimes be less reliable in long interactions," as parts of a model's safety training "may degrade."³ No federal law requires anyone to test for that failure.
3. The fine print decides. A Texas parent told a Senate panel that Character.AI sought to force the family into arbitration under terms the company said the child accepted at 15, terms the parent said cap its liability at $100.⁷

Why legislation: A bipartisan coalition of 44 attorneys general warned AI companies in 2025: "If you knowingly harm kids, you will answer for it."⁸ Senators Durbin and Hawley have proposed applying product-liability standards to AI systems.⁹ Tort law may already reach some of these harms, but no statute says so, so each injured family must first win the argument that a chatbot is a product at all. A company that invites this much trust owes ordinary care in return.

The Solution

A four-step staircase: each step stands alone, and each step up adds accountability. Scope: developers and operators of consumer chatbots, with record-keeping scaled to size and risk and heightened care for products offered to children or marketed for emotional support. Unlike duties aimed at social media and minors or at severe cyber, biological, and physical risks, this one protects a chatbot's own users, of every age.

Step 1 — Preserve the record. Developers and operators keep safety tests, material design changes, and incident records, and preserve specific conversations once a credible claim or legal hold arrives. Courts review them under protective orders; no one's chat logs become public. A family cannot prove a design failure the company did not keep.

Step 2 — Void the fine print. Bar terms that waive the duty, cap damages, or force injury claims into arbitration, as Congress did for sexual-harassment claims in 2022¹⁰ and as the AI LEAD Act would for AI harms.⁹ Firms may still divide costs among themselves by contract.

Step 3 — Write the duty down. Developers and operators must take reasonable care to prevent foreseeable serious physical injury, clinically significant psychological injury, and other defined harms: test before release, evaluate long conversations, respond to credible incidents, and fix known dangerous patterns, judged against reasonable alternative designs. Developers answer for the model; operators, for the persona, memory, interface, and safeguards. Meeting a recognized standard is evidence of care but grants no immunity.

Step 4 — Open the courthouse door. Give injured users a federal claim. They must prove breach, a legally cognizable injury, and factual and proximate causation; the statute defines foreseeable misuse, comparative responsibility, and damages, and a poor answer alone is no injury. Stronger state remedies survive.

Where to start: Step 1 is the floor; it asks companies to keep what they already have. Step 3 is the heart of the proposal.

Administration and enforcement: The duty applies to conduct beginning 18 months after enactment. The FTC, consulting NIST, issues guidance on reasonable practice without deciding individual cases. Federal courts hear private claims, and the FTC and state attorneys general seek injunctions against systemic violations.

Risks and Mitigations

• Causation: A chatbot's presence in a troubled life does not prove it caused harm. Plaintiffs keep the burden of proof, experts test the link, and preserved records keep the inquiry honest; some real harms will still go uncompensated.
• Free speech: Brown v. Entertainment Merchants Association (2011) warns against treating child protection as a license to restrict ideas.¹¹ The duty reaches design choices such as memory, persona, escalation, and testing, the line the Florida court drew between app defects and "ideas or expressions."¹ Other courts may draw that line differently.
• Incumbent advantage: Liability costs weigh most on startups. Scale record-keeping to size and risk, allow insurance, and avoid certifications only large firms can afford; some risk of entrenchment remains.

Similar Bills

Fit measures similarity to this proposal's mechanisms: High = direct precedent; Partial = useful component with material differences; Related = adjacent approach.

Federal — 119th Congress

State

What this adds: The AI LEAD Act supplies a liability architecture, and Maryland shows states moving to treat chatbots as products. This proposal adds evidence preservation, a bar on waivers and forced arbitration for injury claims, and a duty written for conversation itself: testing long chats, fixing known failures, and dividing responsibility between model developers and the apps built on them.