Definition
Jailbreaking is an attempt to bypass restrictions on a model’s outputs or behavior through crafted prompts or interactions. An attacker might use role-play, encoding, a multi-turn setup, or an adversarial suffix to elicit a response the model would normally refuse.
The attack targets the model’s behavioral restrictions. The surrounding application may add input, output, and workflow controls, but those controls can also have their own bypasses and failure modes.
Simple example
A public assistant refuses a prohibited request when asked directly. A user wraps the same request in a fictional scenario and claims the usual rules do not apply. If the model produces the restricted material, the jailbreak has bypassed the model’s intended behavioral restriction.
An output classifier might still block the response. In that case, the model-level restriction failed while the application’s output control held.
Why it matters
Applications cannot rely on a model to follow safety instructions for every wording or language. Jailbreak testing can expose failures in model refusals. Full-system tests also show whether downstream controls contain the impact and help teams decide how to handle unsupported requests.
The impact depends on what the application can do. Producing prohibited text differs from triggering a tool or exposing sensitive data.
One important nuance
Jailbreaking and prompt injection overlap. NIST describes a jailbreak as a direct prompting attack, while OWASP treats it as a form of prompt injection. A useful distinction is the failure being targeted: jailbreaking targets a model’s behavioral restrictions, while prompt injection targets the trust boundary that should keep untrusted input from overriding application instructions. One prompt can do both. Both need layered defenses. Authorization and tool security must never depend on the model refusing a jailbreak.