What is Jailbreak?
SecurityManipulating a model into ignoring its safety instructions. For chatbots it produces bad output; for agents it produces bad actions, which is why enforcement at the tool boundary matters more than instruction hardening.
Why It Matters
A jailbreak convinces the model to set aside its safety instructions: role-play framing, hypothetical contexts, encoded requests, the steadily growing craft of getting the model to want to help with what it was told to refuse. For a chatbot the stakes are a bad answer. For an agent the stakes are a bad action, because the jailbroken model is not just generating text but emitting tool calls with real authority behind them. The chatbot jailbreak produces something embarrassing to screenshot; the agent jailbreak produces a mutation against production.
Jailbreak vs Prompt Injection
The two get conflated and the difference matters for defense. A jailbreak attacks the model’s safety training directly, usually through the user’s own input, aiming at what the model will say or do. Prompt injection smuggles instructions through the agent’s context (documents, tool outputs, other people’s content), aiming at what the agent will do with its tools. Jailbreaks subvert the model; injections subvert the run. An agent can be jailbreak-resistant and still fully injectable, because the defenses sit at different layers.
Where Defenses Break
The natural response is hardening the instructions, and it fails as a boundary: the jailbreak’s whole mechanism is getting the model to set instructions aside, so more instructions just lengthen the text the attack has to defeat. The defenses that hold are structural. Enforcement at the tool boundary denies the action the jailbreak was trying to produce. Blast radius scoping means a fully jailbroken agent still cannot reach what its credentials do not cover. The model is the attack surface; the runtime is the defense.
How Flytebit Handles It
We assume the model can be talked out of its instructions and design accordingly: the governance layer sits outside the model’s reasoning, so a jailbroken model can want to do the wrong thing without being able to. The incident patterns are in Agentic AI Horror Stories, and the control architecture is our AI governance and risk work.