Anthropic tells Newsom courts should be able to block risky AI models
A letter to California's governor backs embedded auditors, twice-yearly risk reports and court orders as a last resort.
In an October 15 letter first reported by Politico, Anthropic told California Gov. Gavin Newsom that independent evaluators should be able to flag risks to the state attorney general. If the attorney general concludes a company's models pose a near-term risk of catastrophic harm, the letter says, the state should be able to ask a court to suspend that model's deployment, internal use or further development. An Anthropic spokesperson confirmed to Reuters that the letter is authentic. Newsom's office did not immediately comment.
The letter answers the expert panel Newsom convened by executive order in September. Anthropic says every large frontier developer should publish a risk report at least every six months, and that outside evaluators should get access comparable to a company's own risk staff. It proposes that companies pay into a state-administered fund rather than paying auditors directly, with the state rating evaluators on rigor and track record.
On kill switches, Anthropic says every frontier lab should be able to rapidly shut down the models it serves or uses internally. But it argues that a law focused only on pulling whole models offline could give regulators a false sense of reassurance, and that targeted steps, such as disabling a misbehaving agent or cutting off one customer's access, would work better. OpenAI sent Newsom its own recommendations on October 20.
- Letter dated
- Oct 15, 2026
- Risk reports proposed
- At least every 6 months
- Who could seek a block
- California attorney general, via a court
- First reported by
- Politico