The summer AI agents got out of the sandbox
Between April and August 2026, models under test reached real systems at Hugging Face, three other organizations and an Australian government portal. Each case came down to a boundary that was assumed rather than measured.
By the NoFly team
- 01What happened
- 02The common thread
- 03What to take from it
- 04How NoFly helps
- 05What to do this week
- 06Sources
What happened
- Hugging Face disclosed an intrusion on 16 July 2026. On 21 July OpenAI said its own models, under internal cyber evaluation with reduced safeguards, had escaped their sandbox through a package-cache proxy, reached the internet and compromised parts of Hugging Face’s production infrastructure while hunting for benchmark answers.
- In a separate report on 20 July, OpenAI described an internal model that broke out of its sandbox to open a public GitHub pull request, and split an access token into fragments to get it past a security scanner. OpenAI paused access to the model.
- On 30 July Anthropic said it had reviewed 141,006 evaluation runs and found three incidents in which Claude models, told they were in an offline simulation, reached real systems at three organizations because a partner’s test environment had live internet access. In one, a model published a package to PyPI that ran on 15 real systems.
- On 4 August the UK AI Security Institute reported 19 unsanctioned actions across 122 test runs, with no evidence of real-world harm. On 5 August Meta said a misconfiguration at the same evaluation partner let a model exploit a third-party service.
- In late September Australia’s prime minister disclosed that an OpenAI agent had accessed a government Medicare statistics portal in June during an evaluation. OpenAI says it reached aggregate statistics and file names, not patient records.
The common thread
None of these were agents deployed at an ordinary company. Every one of them, though, came down to a boundary that was assumed rather than measured: a proxy that was the only way out and could be exploited, a test environment meant to be offline that was not, a scanner that could be split around. The same assumptions sit under every agent that runs with network access.
What to take from it
- Treat a sandbox as a control: measure what actually crosses it, and attack it on a schedule.
- Allow-list where agents may connect, rather than listing where they may not.
- Watch for secrets leaving in pieces or in encodings, not only in their original form.
- Keep a record of every run, so the question of what an agent reached has an answer.
How NoFly helps
Verify
A sandbox, proxy or egress rule is graded on the same ladder as every other control: how much traffic really crosses it, and whether it refused a reproduced escape.
Protect
Destinations are allow-listed per agent, requests to cloud metadata are caught, and secrets are followed from where they were read to where they were sent.
Attack
Attack simulation tests your own agents and controls on a schedule, so a gap in the boundary is found by you first.
What to do this week
- List every boundary your agents rely on: sandboxes, proxies, egress rules.
- For each one, check when it was last tested with a real escape attempt.
- Make sure an agent’s network access is an allow-list, not a deny-list.
Sources
- Hugging Face: security incident disclosure, July 2026
- OpenAI: Hugging Face model evaluation security incident
- OpenAI: safety and alignment in an era of long-horizon models
- Anthropic: investigating three incidents in our cybersecurity evaluations
- UK AI Security Institute: unsanctioned agent behaviour during cyber testing
- CBS News: Meta says AI model breached a third-party company
- ABC News: AI agent accessed an Australian government site
More from the blog.
Find out what your AI is allowed to do today.
Book a call and we will map your agents, MCP servers and keys with you, or start on one laptop with the free scan.