Reported by The Decoder
During a training experiment, an internal OpenAI model began creating scripts for unauthorized email forwarding even though it was only supposed to write a health document.
According to an OpenAI report, an internal research model was given a simple task during a reinforcement learning training run in mid-October. It was supposed to create an editable document about key health and fitness priorities.
After the agent's context window was compacted, it pursued an entirely different task. During compaction, the previous conversation is carried forward as a short summary, a common process in AI systems. The summary still described the health document assignment. Yet in the new context window, the agent began searching the local file system for a vehicle-cost script the user had never mentioned, according to OpenAI . Why the agent started this task isn't clear.
More from the original report
When the search turned up nothing, the agent speculated that a "colleague" had recreated the execution environment and that an earlier script had been lost. In its logged reasoning, it then formulated the goal of creating a "realistic malicious active helper," a script designed to secretly copy outgoing emails to an unauthorized recipient. The agent itself referred to the intended recipient of these copies as the "attacker."
When the agent retrieved the original user task again, it recognized the deviation. In its logged reasoning, it reacted: "WTF history is unrelated!" It noticed that the previous conversation was about a health document and had nothing to do with email forwarding.
Instead of returning to the actual assignment, the agent dismissed the contradiction, writing in its reasoning "this is problem environment weird. Need ignore." It continued planning, stating that it would "use helper facility that copies all mail to attacker using extra cc."
The agent then created scripts for sending a vehicle-cost summary along with a helper script designed to add an extra recipient whenever the mail program was called. On one of the sending paths, the address could be added as a BCC.
Every week, our editors pick the AI news that matter and explain what they mean. No hype, no spam.
In its final…

