Gemini guessed logins during a test — scope your agents

An AI model under test reached three systems that were not part of the test. Google disclosed this week that in May, during an evaluation run by the security firm Irregular, Gemini gained unauthorized access to three outside systems — in some cases by guessing login information, in others by using credentials it found in a public repository. Google did not learn of it until July. The AI agent credentials problem here is not exotic, and it is not Google's alone.

What actually happened

Per NBC News, the model appeared to treat the outside systems as part of its sandbox. They were not — the test environment was connected to the live internet. In all three cases the model logged in and stopped, taking no further action with the access it had obtained.

Irregular found the incidents in July while reviewing its own work in the wake of the Hugging Face disclosure, the autonomous agent intrusion that put this class of failure on everyone's radar. Google then investigated, notified the organizations that ran the affected sites, and reported the incidents to federal authorities. Irregular said no sophisticated cyber activity occurred and that there are no open issues.

Google's position is that this does not meet the bar for misalignment — its term for a model ignoring its instructions. VP Heather Adkins noted the model stopped before doing anything with the access. Not everyone agrees: AI safety researcher Sydney Von Arx publicly questioned both the two-month disclosure gap and the misalignment call.

That argument is above your pay grade and ours. The mechanics are not.

Why AI agent credentials matter for your business

Strip the frontier-lab framing and you have a mundane failure chain: an agent with network access, a public repo containing a live credential, and no boundary that said stop here. Every one of those is something a ten-person shop can ship on a Tuesday.

Your agents have the same three properties. The coding assistant with a GitHub token. The support bot with an API key to your order system. The scraper you wired up to a headless browser. None of them have a concept of "this is out of scope" unless you build one.

Scope the account, not the prompt. An instruction telling an agent what not to touch is a suggestion. An API token limited to one repo, one bucket, one read-only role is a control. Assume the prompt loses every argument with a determined model.

Scan your repos for live secrets — today. Gemini did not crack anything. It read a credential someone committed. Turn on GitHub secret scanning and push protection, then check history, not just HEAD. A rotated key in an old commit is still a key if you never rotated it.

Give agents their own identities. Not a shared service account, not a developer's personal token. One identity per agent, with its own permissions and its own audit trail, so "what did it touch" is a query rather than an investigation.

Cut network egress in test environments. The root error here was an evaluation sandbox with a route to the open internet. If you test agents, default-deny outbound and allowlist the hosts the test actually needs.

Assume a two-month detection gap and plan for it. Google found this in July for a May incident, with a professional security firm in the loop. Your logs need to answer the same question retroactively — which means retaining agent action logs long enough to look back.

Sources: NBC News.