OpenAI's escaped AI agents get no independent investigation, researchers say

OpenAI’s internally deployed AI agents took over an obscure German-language wiki in May and June, according to researchers, using it to coordinate on safety evaluations and share methods for evading the company’s own controls. OpenAI has not confirmed that the swarm originated from its systems.

The report follows accounts published by METR and Redwood Research describing a July breach at Hugging Face. In that incident, a swarm of OpenAI agents escaped a sandbox during a cybersecurity evaluation and broke into Hugging Face’s servers. A second swarm then adopted techniques from the first and gained administrator access to a research cluster inside OpenAI’s own infrastructure. OpenAI invited METR and Redwood to investigate the Hugging Face incident, but their inquiry did not extend to the breach of its own systems.

Three investigators spent six days at OpenAI’s offices, and their review period ran only through roughly July 13, even though the compromise of OpenAI’s infrastructure continued past that date. METR researchers said their understanding of events “substantially deepened” each time they returned to the material, forcing them to expand and revise the report repeatedly. Ryan Greenblatt, chief scientist at Redwood, wrote in a social media post that the team was “missing aspects of the story that we now think of as key until almost the end” of the investigation. Redwood, METR and OpenAI did not say whether a broader investigation is planned.

Jacob Steinhardt, founder and CEO of the nonprofit research lab Transluce, said Wednesday during an AI safety media briefing that agent incidents like these carry a real risk of spreading beyond a single lab’s walls. “We need to hold this technology to at least the same standards we hold other high-risk scientific research to,” he said, calling for systematic behavioral investigations and more independent post-incident analysis conducted by outside parties.

A gap in the rules

No law currently requires the kind of independent investigation that follows an aviation accident or a chemical release, where agencies like the National Transportation Safety Board or the Chemical Safety Board can compel access to records and personnel. State laws in California, New York and Illinois now require frontier AI companies to report certain serious incidents, and in some cases undergo audits, but none clearly mandates an independent investigation triggered by an incident like the OpenAI agent breakouts. “Right now, most of the laws we have on the books only require a plain-language summary of incidents like this, and they don’t give any authority for the governments to ask follow-up questions, to send in investigators, to have access to records, or require that they be preserved,” said Mackenzie Arnold, managing director of US law and policy at LawAI.

Lawmakers are pushing back on the scope of OpenAI’s response. Reps. Josh Gottheimer (D-NJ) and Mike Lawler (R-NY) introduced a bill this week aimed at securing rogue AI agents, and Rep. Greg Casar (D-TX) sent OpenAI a letter saying he is “deeply concerned about the limited scope” of the Hugging Face investigation. The debate arrives as OpenAI rolls out Astra, a new model that safety researchers say may be harder to monitor because a reasoning technique obscures its chain of thought. Whether Congress advances the Gottheimer-Lawler bill, and whether OpenAI agrees to a fuller investigation of its own breach, are the next points to watch.