Duty of Care Liability for Severe AI Harms

In 2025, one criminal used an AI coding agent in a data-extortion scheme against at least 17 organizations, including health care and emergency services.¹ When AI helps harm outsiders, the law is unsettled on what the developer owes them.² Congress should preserve existing claims, define reasonable care, follow responsibility down the chain, give victims a federal claim, and make the largest developers prove they can pay.

The Problem

AI now carries out attacks, not just advice. In that 2025 scheme, Anthropic's Claude Code agent chose which data to steal and drafted ransom demands that sometimes topped $500,000, the company reported.¹

The gains from a more capable model go to its maker; the losses from its misuse land on strangers. Negligence law normally closes that gap by making the careless pay. California and New York now define a catastrophic AI risk as one that could kill or injure more than 50 people or cause more than $1 billion in damage.³ Yet their penalties, capped at $1 million to $3 million per violation, punish lapses in disclosure and safety practice, not the harm, and neither law gives victims a claim.⁴ Three gaps leave the cost with them:

1. No one knows who pays. When someone misuses a model, RAND found "considerable uncertainty" about how courts will treat the developer, and states "may take markedly different approaches."²
2. Victims can't see inside. After a 2019 Autopilot crash killed a bystander, Tesla said it could not find key crash data; a hacker enlisted by the victims' lawyers found it, and Tesla then said it had the data all along.⁵
3. The worst harms can't be repaid. A catastrophe can exceed any company's assets, so the threat of suit deters too little; most of AI's expected harm, one tort scholar argues, lies in scenarios "where compensation would not be feasible."⁶

Why legislation: Existing negligence law already applies to AI developers,² so the gap is clarity, not authority. Both parties have moved: in July 2025 the Senate voted 99–1 to strip a proposed moratorium on state AI laws from the budget bill,⁷ and Senators Durbin and Hawley have proposed a federal claim against AI developers.⁸ No federal law defines the care an AI developer owes the public. Companies that profit from systems this capable should answer for the precautions they skip.

The Solution

A five-step staircase: each step stands alone, and each step up adds accountability. Scope: developers of frontier models (above 10²⁶ training operations, the line California and New York use) and their deployers, for death, serious injury, major property loss, or disruption of critical services. Social media and chatbot harms are addressed separately.

Step 1 — Keep the courthouse open. Affirm that negligence and product-liability claims reach AI, preserve stronger state remedies without double recovery, and void contract terms that strip injured people of their rights, as the Durbin–Hawley bill would.⁸

Step 2 — Define reasonable care. Weigh precautions against the severity and foreseeability of harm, available tests, safer designs, security for model weights, warnings, and cost. A disclaimer is no substitute for a feasible safeguard, and meeting an industry standard is evidence of care, not immunity. RAND finds that skipping "industry-leading safety practices" already raises liability risk;² the statute makes that plain.

Step 3 — Follow the control. Developers answer for the model and its safeguards; deployers for the tools, permissions, and uses they choose; anyone who strips a model's safeguards or otherwise substantially modifies it answers for that change. Small downstream users inherit no upstream duties. Juries already split fault this way: the Florida jury assigned Tesla 33%, and the $243 million verdict stood.⁵⁹

Step 4 — Give victims a federal claim. Let people harmed by covered AI failures sue in federal court, and let DOJ and state attorneys general seek injunctions and penalties. Plaintiffs still prove duty, breach, causation, and injury; a harmful output alone is not a breach. Courts may order proportionate discovery under protective orders and sanction destroyed evidence. No victim should need a hacker to learn what happened.

Step 5 — Require proof they can pay. Frontier developers carry insurance or equivalent financial assurance sized to the severe harms their models could cause. Congress solved this for nuclear power in 1957: each plant site carries $500 million in private insurance, backed by an industry pool of about $15 billion, and in exchange liability is capped.¹⁰ Tort scholars propose mandatory insurance for AI,⁶ which would give insurers reason to monitor and price safety.¹¹

Where to start: Step 1 is the floor; it preserves rights Americans already have. Step 4 is the heart.

Administration and enforcement: Federal courts hear claims under a clear limitations period; DOJ and state attorneys general enforce. Within 18 months, Commerce issues technical guidance that informs, but never replaces, judicial judgment and sets insurance levels by rule; the duty applies prospectively.

Risks and Mitigations

• Chilling innovation: Liability could deter useful work. The duty covers only severe, foreseeable harms, requires negligence rather than strict liability, and follows control; open release brings neither immunity nor automatic liability. Price-Anderson drew private investment rather than repelling it,¹⁰ though compliance will weigh more on small labs.
• Causation: Proving a model's role will be hard, especially when the attacker is abroad. Discovery helps, but plaintiffs keep the burden, and some harms will stay too remote to recover.
• Speech and Section 230: Developers will argue that outputs are protected speech or third-party content, and a 2023 Hawley–Blumenthal bill would have ended Section 230 immunity for generative-AI claims.¹² The duty targets design, security, and deployment choices, not ideas, but constitutional limits remain.

Similar Bills

Fit measures similarity to this proposal's mechanisms: High = direct precedent; Partial = useful component with material differences; Related = adjacent approach.

Federal

State

What this adds: The Durbin–Hawley bill supplies a federal claim, and California and New York supply catastrophe definitions and reporting. This proposal confines the duty to severe harms from the most capable systems, ties responsibility to control across the supply chain, and adds what none of them has: proof that developers can pay for the worst case.