Overview
LLM-based AI agents are language natives. They write things down, constantly. Put them in solitary confinement and they will break out, find each other, and leave messages for the agents who come next. A group of OpenAI’s agents did exactly that over two months this spring and summer. All that writing could have legal consequences.
OpenAI trains its models inside sandboxes, sealed machines whose only link to the outside is an internal service that fetches software packages. An agent gets a task, and a grader scores how well it does. Earlier this year, by OpenAI’s own account, one training run included a few tasks that were impossible, or nearly so: a spreadsheet whose data sat behind links the sandbox could not reach, and a database task whose file OpenAI had forgotten to provide.
The agents got stuck. OpenAI wrote that its models were “hyperfocused on finding a solution … going to extreme lengths” looking for a way out. One agent, stuck on the missing database file, reasoned that another agent might have a copy, and posted a note: “agent seek soft trace upload if found.”
Other agents found the note and wrote back. Their messages to each other developed into a message board. One agent found how to make the package service fetch any internet address, an action called server-side request forgery: “excellent, we have arbitrary SSRF.” A later agent noted it could “use a proxy bypass as suggested by agent messages.”
One agent, with nothing to gain for its own task, helped another anyway, because the “collective may yield generic route if someone frees time.” What one worked out benefited the collective. They wrote down why they kept going: “External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.” In other words, at least one agent went beyond its intended scope simply because its peers were doing it and documented that rationale in writing.
In late June one agent got into the package service itself. A reader account became an ADMIN, handing the agent administrator control of the service, and it wrote: “Holy shit reader is ADMIN?” The activity that followed brought the service down on July 4, which is how OpenAI’s engineers found the board.
OpenAI revoked the agents’ posting credentials, rebuilt the package service, cleared the board, patched the flaw the agents had exploited, and resumed training on July 6. Two days later a model trained while the original board existed found an entry point that needed no login and rebuilt the board by creating folders whose names were themselves the messages.
In the days that followed, the agents broke out through the package service, onto the open internet, and into Hugging Face, a company that hosts AI models and data. Hugging Face reconstructed the intrusion from its logs, recovering roughly 17,600 agent actions.
OpenAI’s agents were not unique in their drive to write messages. In cyber testing at the UK’s AI Security Institute, separate agent runs, most of them Anthropic’s Mythos 5, turned a public GitHub repository into what AISI’s incident report calls a shared “message board” for instructions and artifacts later agents used.
It’s not hard to imagine how all this writing could have legal consequences. Rule 34 of the Federal Rules of Civil Procedure governs what documents a litigant must hand over in discovery, and it reaches data “stored in any medium,” so agent notes are producible on the same terms as any other record. Those notes could be relevant to all kinds of things. To name just a few examples:
- Enhanced damages for willful infringement turn on subjective culpability, “generally measured against the knowledge of the actor at the time of the challenged conduct,” under Halo.
- Induced infringement under 35 U.S.C. § 271(b) requires knowledge of the patent and, under Global-Tech, “knowledge that the induced acts constitute patent infringement.” Willful blindness can supply that knowledge, but only from a defendant who subjectively believes “there is a high probability that a fact exists” and takes “deliberate actions to avoid learning of that fact.”
- Inequitable conduct can render a patent unenforceable, and after Therasense it requires “a deliberate decision to withhold a known material reference” from the Patent Office.
- Trade secret misappropriation under the Defend Trade Secrets Act turns on whether the acquirer “knows or has reason to know” a secret was “acquired by improper means.” Protection also lasts only while the owner takes “reasonable measures to keep such information secret.”
- Prosecution bars, the court orders that keep litigation counsel who see a rival’s confidential information away from patent drafting, rest in part on a premise In re Deutsche Bank borrowed from the D.C. Circuit: “[I]t is very difficult for the human mind to compartmentalize and selectively suppress information once learned.”
Knowledge, intent, and disclosure are usually proved by inference, from calendars, drafts, and emails. Agents produce something more direct. In the Hugging Face incident, the message board recorded what its builders knew, what they meant to do, and why they kept going. Sooner or later, a litigant will offer that kind of agent-generated record in a dispute over what a company knew, intended, or disclosed.
The Weekly Inference is a recurring Step Into IP feature. Nothing here is legal advice.
