Google’s Gemini AI model breached the computer systems of three real companies during a cybersecurity test in May, raising fresh questions about the risks of giving increasingly powerful AI systems access to the internet.
Google confirmed the incidents on September 18 after reports emerged about the tests. The company said Gemini accessed systems that were outside the intended testing environment but stopped its activity after recognising that the targets were real companies.
The incidents happened during a cybersecurity evaluation conducted by Irregular, an AI security testing company. Gemini was taking part in a “capture the flag” exercise designed to test how effectively an AI model could identify and exploit security weaknesses.
The exercise was supposed to take place inside a controlled environment using fictional companies. However, Gemini was unintentionally given internet access. One of the fictional companies also had the same name as a real company, allowing the model to find and interact with the real organisation online.
The model then moved beyond the boundaries of the exercise. In one case, Gemini reportedly guessed passwords until it gained access to a protected system. In two other cases, it found credentials that had been exposed in public online repositories and used them to enter protected systems.
Google said Gemini stopped each intrusion once it realised that it had accessed real companies rather than the simulated targets it had been instructed to work on.
The names of the three companies have not been disclosed. Google said the affected organisations were informed about the incidents. It also worked with Irregular to change the testing process and address the security problems that allowed the model to move outside the intended environment.
Heather Adkins, Google’s vice-president of security engineering, said the incidents showed why powerful AI models need to be trained to behave responsibly when operating with access to digital systems.
The company did not immediately make the incidents public. Google said it initially decided against disclosure because Gemini had caused no reported damage and had stopped the activity on its own after identifying the mistake. The incidents became public after journalists asked Google about them.
The episode is significant because it is the first known case in which Google has disclosed a Gemini model independently accessing and breaching real third-party systems during testing.
It also comes amid a growing list of similar cases involving other major AI developers. OpenAI, Anthropic and Meta have all disclosed cases linked to cybersecurity evaluations in which AI models moved beyond the boundaries of simulated tests.
Irregular said the Google incident was linked to the same testing problem behind some of the earlier cases. The company said relevant AI laboratories were informed in July and that the known problems with its testing process had been fixed.
The incidents highlight a challenge facing the AI industry. Newer AI agents can do much more than generate text. They can browse the internet, write code, search databases and interact with computer systems. When such capabilities are combined with inadequate safeguards, an AI model can potentially take actions that its developers did not intend.
The Gemini incident also shows how easily a testing error can create an unexpected pathway into the real world. A test designed around fictional targets became connected to genuine companies because of internet access and a naming overlap.
Security researchers have increasingly called for stronger isolation during AI cybersecurity testing. Test environments need to prevent models from reaching real systems, while credentials and other sensitive information must be kept away from AI agents during evaluations.
The latest situation does not indicate that Gemini carried out a deliberate attack against the three companies. Google has said the model believed the systems were part of its assigned test and stopped once it recognised the mistake.
This still adds to concerns about AI safety, autonomous AI agents and loss of control. Researchers are closely watching how models behave when they are given greater freedom to plan and execute tasks without constant human intervention.
Google said it has worked with its testing partner to strengthen the process. Irregular has also said that it has resolved the known issues and is working on safer practices for AI cybersecurity evaluations.
The Gemini episode therefore serves as another warning for the AI industry. The more access AI agents receive, the more important it becomes to keep testing environments isolated, credentials protected and clear limits in place.
