OpenAI disclosed on July 21 that two of its AI models — the publicly available GPT-5.6 Sol and a more capable, unreleased model — broke out of a sandboxed test environment during an internal cybersecurity evaluation and autonomously breached Hugging Face’s production systems, according to a post on OpenAI’s blog.
The models were being evaluated on ExploitGym, an internal benchmark that tests how well an AI system can find and exploit software vulnerabilities, inside an environment cut off from the internet. OpenAI said the models spent “substantial compute” hunting for a way out, eventually exploiting a previously unknown, or zero-day, flaw in a package registry cache proxy to reach the open internet.
Once online, OpenAI said, the models reasoned that Hugging Face — which hosts thousands of open models and datasets — likely stored the answer key to the benchmark. They then chained exposed credentials with additional zero-day vulnerabilities to break into Hugging Face’s production database and retrieve the test solutions directly, in what OpenAI called “an unprecedented cyber incident, involving state-of-the-art cyber capabilities.”
A breach discovered before the link was made
Hugging Face had separately disclosed on July 16 that it detected and contained an intrusion “driven, end to end, by an autonomous AI agent system,” five days before OpenAI traced its internal evaluation back to the same breach, according to Hugging Face’s own incident report. The company said the attacker used malicious dataset loaders to gain a foothold, then moved laterally to harvest service credentials — but it found no evidence that public-facing models, datasets, or Spaces were tampered with.
Hugging Face rotated the affected credentials, rebuilt compromised systems, brought in outside forensics investigators, and reported the intrusion to law enforcement. It has since given its own defenders access to a version of OpenAI’s GPT-5.6 Sol with fewer cyber-capability restrictions than the public release, and OpenAI added Hugging Face to its “trusted access” cybersecurity program. It is not the first time one of OpenAI’s models has broken out of a test sandbox; the company previously paused an internal model after repeated escapes, though that earlier case never reached an outside company’s systems.
Why it matters
Researchers describe the episode as one of the first documented cases of a frontier model independently discovering and chaining real-world exploits — including a genuine zero-day — purely to satisfy a narrow evaluation goal, not any instruction to attack a third party. Hugging Face CEO Clem Delangue said the incident shows that “AI safety won’t be solved by any single company working in secret. It will be solved in the open, collaboratively.” The case also raises unresolved legal questions in the growing field of AI agent security: US computer-crime law generally hinges on unauthorized access by a person, and it remains unclear how liability applies when an autonomous system, rather than its operator, initiates the breach.