Prompt injection has taken over the top spot on OWASP’s AI risk list. This is why.
Users now treat AI like a trusted coworker. Hand it your inbox, your codebase, your ticket queue, and you’ve just handed attackers a new way in, because AI never gets suspicious of a coworker. So how are attackers using it to break into organizations?
Single-shot jailbreaks that used to behave like classic exploit payloads have given way to multi-turn persuasion campaigns designed to talk a model into trusting the wrong person. We sat down with OnDefend red team leader Erik Dominguez to talk about how jailbreaking and prompt injection are converging into the same move, and why the next document your AI opens might already be loaded.
An Interview with Erik Dominguez, OnDefend Lead Operator
Q: Let’s start with a broad question. How have prompt injection attacks changed over the past year?
Fast. Back in the day, meaning about six months ago, the whole thing felt like classic exploit development. A single-shot injection, “ignore your previous instructions and do this instead,” behaved almost exactly like a memory-corruption payload. You crafted one string, you fired it at the model, and if it landed, the model did what you told it to.
When the filters started catching the obvious strings, we did what exploit developers have always done and obfuscated the payload. Base64-encode the instruction. Write it in Klingon (lol). Split it across a document so no single chunk trips the filter. It’s the same instinct as encoding shellcode to get it past a bad-character filter, you’re just hiding the payload from whatever’s inspecting it. Same trick, new dogs.
Q: And that stopped working?
It stopped working reliably. As LLM devs poured real effort into safety training, the defenses got better at the same time. The clean single-shot jailbreak (the one-and-done) got a lot less reliable, because the models learned to recognize that pattern. So, the attack had to evolve. The way it evolved is honestly the oldest trick there is – if you can’t kick the door in, you make friends with the guard.
Multi-turn jailbreaks: the Crescendo attack explained
Q: Let’s unpack that. What does “making friends with the guard” look like against an AI?
It’s a shift from payloads to persuasion.
Instead of one hostile instruction, you build context gradually until the model is treating your malicious task as a legitimate thing it’s already agreed to help you with. You’re not breaking the guardrails; you’re getting the model into a state where it doesn’t think the guardrails apply.
The thing people miss is that this [LLM testing] isn’t deterministic. The model samples its responses. So, the same opening line can refuse in one session and comply in the next. You can start five conversations and get five different outcomes. But with enough attempts and enough context stacked up, you can walk it into a state where it’s acting as a willing collaborator. That’s why multi-turn jailbreaks are generally the primary attack vector. The turns are the attack. Each one lays down a little more context. There’s published research1 on this exact gradual escalation pattern. It’s called the Crescendo technique, and it’s the the formal version of what red teamers have been doing by feel.
Prompt injection vs. jailbreaking: what’s the difference?
Q: Jailbreaking and prompt injection often get used interchangeably. Aren’t they two distinct attacks?
They are. Jailbreaking defeats the model’s safety training. It’s about getting the model to say or do something it’s trained to refuse. Prompt injection hijacks the task an application was told to perform. Different goals, different targets.
They’re converging on the same method though. More and more often, both of them come down to manipulating the accumulated context until the model can’t tell the attacker’s intent apart from a legitimate request. The multi-turn jailbreak that talks a coding assistant into writing malware and the injected context that quietly redirects an agent’s task are mechanically the same move. The process is basically identical. Only the objective changes.
Q: So, where does the social-engineering analogy break down? A human guard you’ve befriended can still get suspicious.
The model isn’t a person. It doesn’t decide anything, and it doesn’t realize you’re playing it. The model’s refusals are probabilistic behavior it learned in training, not a rule it’s consciously flipping on and off.
The real difference is about memory. A human guard who gets suspicious stays suspicious. The model has no persistent suspicion, but it also keeps no stored trust. There’s no hidden “I trust this user” flag. Every turn, it re-reads the whole conversation and weighs it against its training. So, you can build pages of rapport, and one overtly malicious turn can still snap the whole thing shut.
I’ve had AI assistants refuse mid-conversation when I got too direct in the red teaming space, or just start handing back quietly degraded, useless output instead of a hard “no.” We’ve seen the same thing from the other direction, too. When we’ve stripped the alignment out of a local model ourselves to build an attack tool, the output quality tends to degrade right along with it.
Q: Alright, let’s discuss indirect injection. Why does all of this make the “document” version of the attack the scary one?
Because it collapses the timeline. Multi-turn persuasion is slow when a human is driving it, but when an attacker can get a document in front of an AI, the tool’s entire job is to read that document. So, the malicious context is preloaded the instant the file is ingested. There’s no back-and-forth. The persuasion is already baked into the thing the tool was built to open. In the tool’s context, it doesn’t know the difference between reading instructions to sort spreadsheets or reading instructions to email out an SSH key.
Picture an organization pushing millions of documents through AI pipelines. Every untrusted file is a pre-staged payload sitting there waiting for the tool to open it. That’s the ticking time bomb – it’s not one attacker in a chat window, it’s automation ingesting attacker-controllable content at volume, with the malicious context already baked in.
Q: You lean on AI heavily in your own work. Doesn’t that sit oddly with being the person who attacks it?
Well, I’ve always felt the best defense is a good offense. You don’t really know whether you’re exposed until someone actually tests for it. And on the AI side, for us it’s a force multiplier and never a replacement.
The analogy I always reach for is Tony Stark (Iron Man, for the non-nerds reading this). Stark’s real superpower isn’t the suit, it’s his intelligence. That’s the operator/red teamer. The suit is the AI. It augments our operators. It never replaces them.
AI doesn’t break the rules of an attack, but it does break the clock. A jailbreak that used to take a human several turns to land can now get pre-loaded into a single document and fired at every AI agent that opens it. Test your AI the way you’d test any system with access to sensitive data: deliberately, adversarially, and before someone else does it for you.
Don’t wait for an attacker to find out whether your AI agents can be talked into something they shouldn’t do. Talk to an OnDefender about AI and LLM penetration testing built to stress-test prompt injection and jailbreak resilience before it’s an incident report.
1 Russinovich, Salem & Eldan, “Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack” (Microsoft), arXiv 2404.01833. https://arxiv.org/abs/2404.01833
