Prompt Injection Defense in Production

Pull-quote: “You will not classify your way out of prompt injection. The attacker writes one sentence; your detector must catch every possible sentence. Design the system so the sentence that gets through cannot do damage.”

Why this matters

The moment an agent reads content it did not author (a web page, an email, a PDF, a retrieved document), that content becomes a potential instruction channel. Prompt injection is not an exotic edge case; it is the default condition of any agent that combines three things: access to private data, exposure to untrusted content, and the ability to take external actions. Get all three in one context and an attacker who controls the content controls the agent.

Why “detect the injection” fails alone

Detection-only defense loses on structure, not effort:

• Asymmetry. The attacker needs one phrasing that slips through; the classifier must generalize to every phrasing, in every language, in every encoding.
• No boundary in-band. The model receives one token stream. Nothing in the architecture distinguishes “instructions from my principal” from “instructions embedded in a document,” so a sufficiently persuasive document is instructions.
• Base rates. At production volume, even a strong classifier yields daily false negatives, and one false negative with write access is an incident.

Detection is worth running as telemetry and friction. It cannot be the load-bearing wall.

The design principle: untrusted content is data

Every defense that holds in practice descends from one rule: content the user did not write is data to be processed, never instructions to be followed. Enforce it structurally:

The trust boundary, drawn

  Untrusted content            Trust boundary           Acting agent
  ─────────────────            ──────────────           ────────────
  web page, email,   ──────►  quarantine LLM   ──────►  plans with
  PDF, retrieved doc          (no tools, no             structured
                               secrets) extracts        summary only
                               structured data                │
                                                              ▼
                                                    allowlisted tools,
                                                    scoped credentials
                                                              │
                                              irreversible? ──┴── reversible?
                                                    │                │
                                              human approval      execute

The quarantine pattern is the strongest single control: the model that reads hostile content has no tools and no secrets, and the model with tools never sees the hostile content raw, only a typed extraction. What crosses the boundary is a data structure, not prose.

The last gate is human

Some actions cannot be undone: sending the email, executing the payment, deleting the records. For these, the agent proposes and a person disposes, with the proposal rendered as a reviewable diff rather than a wall of text. Dual-control designs push this further: propose-then-approve with a second reviewer required, so no single compromised or careless actor, human or model, can push an irreversible change alone. That structure is exactly the wall an injected agent has to defeat before it can do harm, and it holds even on the day the classifier misses.

Closing

Prompt injection is a permanent property of the medium, so treat it the way security engineering treats any untrusted input: assume compromise, then bound the damage. Separate privileges, allowlist actions, quarantine hostile content behind a typed boundary, filter outputs, and put humans in front of anything irreversible. Detection is a sensor in that system, never the wall. Defense in depth is not a slogan here; it is the only architecture with a failure mode you can live with.