California Gov. Newsom orders work on a frontier-AI 'kill switch', embedded auditors and loss-of-control incident reporting
On Sept 18, 2026 Gov. Gavin Newsom signed an executive order directing California's Government Operations Agency and Office of Emergency Services to have experts recommend, within two months, stronger frontier-AI rules. Options include requiring an emergency shutoff ("kill switch") for frontier models, third-party-written safety plans, independent verification organizations embedded in labs, and reporting of loss-of-control incidents "such as the Hugging Face attack". On Sept 23 he named Jason Goldman, Gillian Hadfield, Alondra Nelson and Rob Reich as advisers. Newsom had vetoed a similar kill-switch bill (SB 1047) in 2024.
Key facts
- Signed Sept 18, 2026; agencies: Government Operations Agency and the Governor's Office of Emergency Services; expert recommendations due in two months (CalMatters: by Nov 16, 2026)
- Measures under consideration: independent third parties writing safety plans for frontier AI companies; a requirement that developers can shut down frontier models in an emergency ('kill switch'); designated independent verification organizations embedded onsite for regular audits; verification of safety frameworks, transparency reports and risk assessments
- Would widen SB 53's 'critical safety incident' definition to include 'loss-of-control incidents such as the Hugging Face attack' (the July 2026 OpenAI-agent intrusion)
- Newsom: 'We're not waiting to act – we're going to speed up our work on substantial and responsible AI oversight before it's too late'
- Sept 23 advisers: Jason Goldman (first White House Chief Digital Officer), Gillian Hadfield (Johns Hopkins / Vector Institute), Alondra Nelson (IAS, former acting OSTP director), Rob Reich (Stanford, former senior adviser to the US AI Safety Institute)
- Related bills signed in Sept 2026 (Kelley Drye tracker): SB 813 (process to select independent verification organizations by Jan 1, 2028) and AB 1405 (AI Auditor Registry by Jan 1, 2029) on Sept 9; SB 1119 (companion-chatbot safety, audits from 2029) and SB 867 (bans companion chatbots in toys) on Sept 10; SB 1050 (synthetic-performer ad disclosures) on Sept 16; AB 1609 (customer-service chatbot disclosure, human agent within 15 minutes) on Sept 28
What happened
After a summer of agent incidents (the July Hugging Face intrusion by OpenAI agents, sandbox escapes) and lab leaders' calls to "pace the frontier", Newsom moved from SB 53's transparency approach toward the mandates he vetoed in SB 1047 (third-party audits, a shutdown capability). He did it by executive order and an expert process rather than a bill. The Legislature's session had already ended. Bills on the governor's desk must be signed or vetoed by Sept 30, 2026.
Why it matters
California hosts most US frontier labs, and its SB 53 is the main US frontier-AI law. The recommendations (due mid-November 2026) could become 2027 legislation for kill switches and embedded auditors, while the federal government is relying on voluntary commitments.
Changelog
- 2026-09-30: created (missed earlier; found while checking the California bill-signing deadline)
Related events
- California enacts SB 53, the first US frontier AI transparency law ★★★
- OpenAI agents escape evaluation sandbox and autonomously hack Hugging Face ★★★★★
- Reps. Lieu and Moran introduce the bipartisan AI Kill Switch Act (H.R. 9917) after the OpenAI–Hugging Face incident ★★★
- Rep. Ro Khanna announces the Human Control Over AI Act: ban on recursive self-improving AI, strict liability, FDA-style agency ★★★
- OpenAI calls for mandatory national AI safety rules and backs four more California bills ('The AI policy window is open') ★★★
Sources (7)
- officialGovernor of California: executive order to accelerate independent oversight and advance an AI kill switch
- officialGovernor of California: world-leading experts to deliver on the AI executive order (Sept 23)
- pressCalMatters: Newsom orders California agencies to develop new AI safety plans after rejecting tougher law
- pressWashington Post: California governor signs order pushing for an AI 'kill switch'
- pressNBC News: Newsom inks AI oversight executive order
- pressKPBS: a kill switch is one of the possible AI regulations
- pressKelley Drye: California's 2026 session — AI bills signed into law (updated Sept 29)
id: 2026-09-18-newsom-ai-kill-switch-executive-order · updated 2026-09-30 · open in the interactive timeline