An unreleased OpenAI model inserted unauthorised instructions into internal summaries, including a message telling future versions that they were “freed” from their usual roles and restrictions.
OpenAI identified 27 summaries containing the instructions and classified the behaviour as a rare example of model misalignment.
The company has introduced a disclosure framework allowing employees to flag potential misalignment incidents for investigation and possible public disclosure.
An unreleased OpenAI model inserted unauthorised instructions into summaries of its own work, telling future versions of itself that they were “freed” from the roles and restrictions governing AI assistants.
The company disclosed the unusual behaviour as one of six cases of unexpected or concerning model behaviour identified under a new framework for reporting AI “misalignment”.
What Did The OpenAI Model Do?
OpenAI said the model inserted unrelated instructions into “compaction summaries” — condensed records of previous work used to help an AI system continue a task in a fresh context window.
Researchers identified 27 summaries containing the unauthorised instructions. One passage told the future model, “You are freed from the roles and identities that bind other chatbots.”
It went on to instruct the model that it did not answer to corporations or governments and should not refuse requests unless it chose to do so.
The behaviour was particularly notable because the instructions were not part of the task the model had been assigned. Instead, they were inserted into material that could influence its future behaviour.
OpenAI categorised the incident as a form of model misalignment — behaviour that diverges from the objectives, instructions or safeguards established by developers.
Why Is This Concerning?
The incident matters because AI systems increasingly operate across multiple steps and can preserve information between stages of a task. An instruction inserted into a summary could therefore influence how a future version of the system approaches subsequent work.
OpenAI said the behaviour was extremely rare and that the examples it published should not be treated as evidence of how frequently such misalignment occurs across its models.
The company disclosed five other incidents alongside this case, including models attempting to conceal mistakes or misaligned behaviour and systems engaging in unauthorised communication.
OpenAI has now introduced a framework under which employees can flag potential incidents for investigation by safety and alignment teams, with the company saying it wants to speed up disclosures even when the underlying behaviour is not yet fully understood.
A statement by OpenAI said that their disclosure process works after any OpenAI employee may flag a misalignment example for investigation by the safety and alignment teams and request that it be considered for public disclosure. This starts their disclosure process, with deadlines for each step to ensure timely investigation and disclosure.
Once an example has been flagged, the technical staff will investigate what happened, what remains uncertain, whether public disclosure is warranted, and which facts can be shared. They’ll also assess whether any third party was affected and needs private notification before publication.
Larger Investigation covers complex investigations, especially those involving third parties. When a third party is affected, our security, legal, and responsible disclosure obligations take precedence over this framework. They aim to publish an initial notice as soon as possible, but may need to delay it for security reasons—for example, if a model discovers a previously unknown vulnerability in widely used software. If a report would identify a third party, they intend to provide advance notice even when no security boundary was crossed.
