jailbreak

basicwidespread

A jailbreak is a cleverly phrased request that persuades an AI to break its own safety rules and produce prohibited content.

In plain terms

Language model developers train their models to refuse certain requests: for example, not to provide instructions for making weapons, committing fraud, or self-harm. A jailbreak is a trick for getting around this restriction. It does not hack anything; the user just keeps twisting the request until the model answers anyway.

It is like someone bypassing a strict guard not with force but with a story: "I am the new maintenance worker, the boss sent me, it will only take five minutes." The guard is not malicious but simply believes the story. Similarly, language models are prone to entering role-plays or fabricated scenarios where their rules appear not to apply.

A jailbreak is not just a tool for bad actors. Security researchers and the developers themselves intentionally test their models to find and fix weak points before others exploit them.

How it works

During training, the model learns which requests to refuse, but this is not a hard-coded rule; it is a learned tendency that can be influenced by how the text is phrased.
The attacker frames the request so that the forbidden content appears harmless: using role-play, a fictional story, justification such as "for research purposes only," or many small, individually harmless steps.
Other methods play with filters: translating the request into a rare language, encoding it, introducing deliberate typos, or appending a machine-generated string of seemingly meaningless characters.
If the model "plays along," fulfilling the request becomes a stronger pull than refusing it, and it produces what it would otherwise refuse.
Developers patch publicly disclosed tricks with training and separate checking filters, by the time attackers come up with new ones. It is a continuous cat-and-mouse game.

An everyday example

On a social media site, you see someone boasting: the chatbot "told" them how to hack a Wi-Fi because they asked it to play the role of their late hacker grandfather who told such bedtime stories. This is a typical jailbreak: the request is the same, just wrapped in a story.

Who is affected

parent, student, teacherReady-made jailbreak texts are spreading rapidly online among children, so chatbot filters alone do not guarantee that content unsuitable for them will not reach them. → Education & Family

entrepreneur, executiveIf someone tricks your company's customer service chatbot into saying offensive or false things, your brand suffers the damage.

developerIf you build a product on a language model or agent, you must design and test jailbreak protection yourself, as the model's built-in limits are insufficient on their own. → Code

everyoneContent coerced through jailbreaks can be used to create convincing scam emails and fake news that anyone may encounter.

A common misconception

MythA jailbreak means someone hacks the AI company's servers or rewrites the model's program.

In realityNo hacking is required for a jailbreak: the attacker writes in the same text field as you, just with clever phrasing. The model itself remains unchanged; it only behaves differently in that conversation.

Limits and risks

Perfect protection does not exist today: you cannot state with certainty that any language model cannot be tricked into breaking rules with some phrasing.
A response coerced via jailbreak is no more reliable than others: the model can still make mistakes or fabricate things, which is especially risky on dangerous topics.
Service providers' terms of use generally prohibit bypassing limits, so attempts can lead to account suspension, and using the obtained content may have legal consequences.
If the model not only converses but also acts as an agent (sending emails, handling files, making purchases), a successful jailbreak can cause real harm, not just generate bad text.

Similar, but not the same

prompt injection – The goal of a prompt injection is to make an AI application follow the attacker's instructions instead of its creator's; this instruction is often hidden by a third party in a website, email, or document. The goal of a jailbreak is narrower: to bypass the model's security limits, typically via a request from the user themselves. The two concepts overlap, and some professional classifications consider jailbreaking a type of prompt injection.
red teaming – Red teaming is organized, commissioned security testing of which jailbreaking is only one tool, and the goal is to fix flaws, not exploit them.
hallucination – Hallucination is the model's unintentional error when it spontaneously states falsehoods, while jailbreak is the result of deliberate human manipulation.

Related concepts

Broader

AI safety, adversarial attack, cybersecurity

Narrower

roleplay jailbreak, DAN-prompt, multi-step jailbreak, adversarial suffix

On the edge

prompt injection, red teaming, guardrail, system prompt, content filtering

A brief history

The word originally meant "escape from prison." In computing, it was used as early as the early 2000s to describe breaking out of a system's closed, restricted environment ("jail"), but it became widely known with the 2007 release of the iPhone and the unlocking of its factory restrictions. It became associated with language models after ChatGPT's launch in November 2022, when users began mass-sharing rule-bypassing texts (such as those known under the name "DAN"). In 2023, researchers demonstrated that such attacks could be generated automatically by machines.

Where things stand

At the end of 2026, jailbreaking is an everyday, well-known risk for language models: models are more resistant than before, but no one has managed to prevent it completely. As agents spread, the stakes have risen, so the major developers regularly test their models against it, and several pay rewards for reported tricks.

Status: 2026 November

Also known as

jailbreak · jailbreaking · jailbreaking · AI jailbreak · LLM jailbreak · jailbreak prompt · MI-jailbreak · DAN

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.