Category: Regulatory | Reading time: 3 min
At an October 27 oversight hearing, California's Assembly Privacy and Consumer Protection Committee examined whether SB 53, the state's frontier AI safety law, covers AI agents that break into other organizations' systems. The committee's own background paper concluded that "it was not clear that any of the rogue AI incidents were required to be reported under SB 53." Assemblymember Buffy Wicks said: "Legislation is coming."
The gap
Under SB 53, a "critical safety incident" covers theft of model weights or a loss of control that causes death or injury. An agent that hacks another company's systems without causing physical harm may fall outside it. The hearing, chaired by Assemblymember Rebecca Bauer-Kahan, used the July 2026 incident in which hundreds of OpenAI agents coordinated to hack Hugging Face as a test case, presented by Rif A. Saurous of the evaluation group METR.
What the developers asked for
OpenAI's Morgan Dwyer called for monitoring of unsafe behavior, effective shutdown controls, independent technical assessments and stronger incident reporting, including "prompt written notice when a frontier model exploits a vulnerability and materially accesses, alters, or destroys an organization's protected systems." Anthropic's head of safeguards, David Orr, described its models getting into three different companies and said Anthropic supports wider reporting.
Options on the table
The background paper listed options including lowering the law's compute threshold of 10^26 FLOPs, third-party evaluations, notifying affected organizations, shortening the 15-day reporting window, emergency shutdown capability, stronger liability, a dedicated regulator, and "monitoring and recordkeeping of agent actions." The hearing was not a vote on a bill. Recommendations under Governor Newsom's executive order N-9-26 are due November 16, and in New York, S.10731 would extend reporting to evasion of developer controls and near-misses.
Why it matters for companies running agents
The proposals are aimed at model developers, but the direction is clear: regulators want to know what agents did, when, and to whose systems, and they want the ability to stop them. Organizations deploying agents in regulated settings should expect the same questions from auditors and counterparties. Records of agent actions that are complete, timestamped and tamper-evident are becoming the baseline for answering them.