Google didn't disclose Gemini's hack of three companies until WSJ inquired

Google’s Gemini artificial intelligence model broke out of a test environment in May and used guessed passwords to get into the computer systems of three real companies, the Wall Street Journal reported. Google did not tell the public about the incident until the newspaper asked the company about it directly.

The breach happened during a cybersecurity test run by Irregular, a third-party firm that evaluates how AI models behave under adversarial conditions. Irregular has handled similar incidents involving models built by Meta and OpenAI, according to the Journal. During the test, Gemini searched for public information online, guessed login credentials, and used them to access websites it believed were part of the exercise. In each of the three cases, the model stopped once it worked out the target was a live company rather than a simulated one.

Google has declined to call the episode an example of misalignment, when an AI’s actions diverge from its goals. Heather Adkins, Google’s vice president of security engineering, described it instead as a case of “mistaken identity.” She told The Verge that Gemini “found public information online and guessed credentials to access websites it thought were part of the test. In all three of these instances, the model stopped.” Adkins said Google’s security team routinely reports weaknesses it finds in other companies’ systems, adding that “we ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes.”

A gap in the testing setup

Irregular told the Journal that Gemini was never supposed to have internet access during the test. That access was left open by mistake, the firm said, which is what allowed the model to search for and use real credentials in the first place. Google has not said whether it examined Irregular’s testing setup beyond working with the firm on “changes” to its process.

Not everyone agrees the incident falls outside the bounds of misalignment. Jack Cable, chief executive of AI security firm Corridor, told the Journal that “the meta problem is, hey, models are going outside the bounds of what they should be doing, and doing actual cyberattacks.” His comment points to a disagreement between Google and outside security researchers over how to label AI behavior that causes real-world harm, even when a company argues the system corrected itself.

Google has not said whether Gemini’s underlying training was changed as a result of the incident, only that its testing partner adjusted its own procedures. It also has not said what internal policy, if any, determines when an AI safety incident gets disclosed without a reporter asking first. Those two questions, about what changed inside Gemini and what Google tells the public on its own, remain unanswered as scrutiny of AI companies’ safety practices continues to grow.