Anthropic pitches AI safety bill that may block a model if 50 people could be killed

What happened

Anthropic drafted a federal AI safety bill that would let the government block a model's release through a court order. Politico reported it first; Anthropic confirmed the draft.

Context and impact

AI labs have favored voluntary commitments until now. This is the first time a major AI lab has formally proposed legislation with mandatory federal court enforcement. The 50-death threshold mirrors California's SB 53.

Details

• Risk must stem from a biological weapon, model evasion of developer controls, or a cyberattack
• Independent evaluators registered with the Commerce Department assess models
• If they find imminent risk: 72 hours to notify Commerce; Commerce then asks a D.C. judge for a halt
• No federal agency can halt distribution on its own — always requires a court order
• Nvidia CEO Jensen Huang has argued no new AI laws are needed