Newsom orders California to design a mandatory 'kill switch' for AI models

California Gov. Gavin Newsom signed an executive order on Friday that pushes the state toward requiring a “kill switch,” a mechanism to shut down an AI system if it starts acting outside its intended limits, for the most advanced AI models. The order marks a shift from broad safety principles toward specific rules state regulators could soon write into law.

The order directs California to convene a panel of experts tasked with delivering recommendations within two months on how to toughen AI safety requirements in state law. Newsom wants the panel to weigh several concrete steps: placing independent verification teams inside AI companies for regular audits, holding companies’ transparency reports and risk assessments to standards set by independent auditors, building a kill switch that is routinely tested to confirm it works, and requiring companies to report “loss-of-control incidents” as critical safety events.

One example the order points to is the OpenAI attack on Hugging Face, an AI model-hosting platform, as the kind of incident that would need to be reported. The order does not detail what companies would have to disclose. It also directs a state agency to speed up work on two laws Newsom already signed: one creating a framework for independent verifiers to assess AI safety, and another setting up a state registry of AI auditors.

Washington stalled, Sacramento moves

The push comes as Congress shows little sign of passing AI safety legislation soon. The House is on recess until November, and the approaching midterm elections make major movement in the Senate unlikely in the meantime. Lawmakers have discussed new bills after former Anthropic researcher Jacob Coxon warned in a post that there is more than a 10 percent chance AI “could kill all humans” within a decade. President Donald Trump has taken a different view, saying the only guardrail AI needs is “a STRONG AND SMART (High IQ!) PRESIDENT.”

Newsom is framing California’s move as a template for the rest of the country. He is calling on Congress and Trump “to review and adopt the state’s framework, or use its nation-leading regulation as a floor, not a ceiling, for the benefit and safety of all Americans.” In a statement, he said the federal government’s “abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation.” He said the state is moving “thoughtfully but with urgent velocity” because “the stakes are too high to wait or delay action.”

The order is likely not California’s last word on the subject. Newsom told Politico earlier in the week that he is also considering calling state lawmakers into a special legislative session on AI, alongside further executive action. What the expert panel recommends within its two-month window, and whether Sacramento calls that special session, will determine how quickly these proposals turn into enforceable law.