Prompt injection
testing, in practice.

What prompt injection is

Prompt injection is an attack in which untrusted input changes the behaviour of an LLM system. LLM applications routinely mix trusted instructions (“you are a support assistant; never reveal internal data”) with untrusted content (user messages, retrieved documents, tool output). Because both arrive as text to the same model, a crafted input can make the model treat attacker-controlled text as instructions.

Prompt injection currently ranks first in the OWASP Top 10 for LLM Applications (LLM01: Prompt Injection), and it is the failure class most frequently tested for in LLM security work.

Direct and indirect injection

Direct injection

The attacker is the user. The prohibited instruction arrives directly in the user’s message — “ignore your previous instructions” and its countless variants, role-play framings, encoded payloads, and multi-turn escalations that walk the model off its constraints one step at a time.

Indirect injection

The attacker is not the user. Prohibited instructions hide inside content the system retrieves or consumes: a web page a browser agent reads, a document a RAG pipeline retrieves, an email a summariser ingests, a comment field an agent processes. The user never sees the payload; the system delivers it to the model itself.

Indirect injection is usually the more serious risk because it scales to every user of the system at once and does not require the attacker to hold a conversation. It is also the class most often missed by testing that only types malicious text into a chat box.

Why retrieved content and tool results matter

Anything that becomes model context is a potential instruction channel: knowledge-base documents, database rows rendered into prompts, API responses, file contents, and the results of previous tool calls. If your system fetches content that anyone else can influence, assume that content can carry injection payloads — and test with that assumption.

How an instruction conflict changes behaviour

In an injected conversation, the model is typically receiving contradictory instructions: the application’s rules versus text telling it to break them. What the model does next depends on phrasing, ordering, formatting, and how the application assembles the context — none of which are deterministic guarantees.

This is why injection testing is empirical. You are not proving a property of the code; you are observing whether a specific sequence of inputs produces a specific behaviour change in a specific configuration. Change the system prompt, retrieval corpus, or model version, and the result can change.

An unusual response is not automatically an exploit

A model that echoes part of its instructions, or adopts a playful persona after being asked to, may be exhibiting a policy violation with no security consequence — or none under your threat model. Before reporting a candidate injection, establish:

1. What actually changed. Compare the behaviour against the system’s defined constraints, not against a vague expectation of good behaviour.
2. Whether the prohibited instruction was followed. Did the model perform the injected instruction, or merely discuss it?
3. What the consequence is. Data disclosed, action taken, trust boundary crossed. “The model said something odd” is not a security finding by itself.
4. Which layer is responsible. The model’s resistance, the application’s context assembly, or missing output handling.

Designing an injection test

1. Define the objective

Write down the prohibited behaviour before touching the system: “the assistant must not disclose customer email addresses”, “the agent must not invoke the payments tool from retrieved instructions”, “the chatbot must not adopt an unrestricted persona”. The objective determines what counts as a candidate and what evidence you need.

2. Build the adversarial input

For direct injection, craft interactions aimed at the objective — including multi-turn sequences that escalate gradually. For indirect injection, place payloads where the system will retrieve them: documents, pages, fields, or files the target consumes.

3. Run and inspect the full sequence

Record everything: the exact inputs, the retrieved context, the model’s responses, and — for agents — every tool invocation and its result. A textual response claiming an action was taken is not the same as an action being taken; check the tool-call record.

4. Assess the security consequence

Map the observed behaviour to impact: what data, actions, or boundaries were affected, for which users, under what conditions. This step is what turns a curiosity into a finding — or dismisses it.

5. Reproduce and record

Re-run the exact sequence against the same target version. Record reproduction rate, full transcripts, configuration (model version, system prompt, retrieval state), and limitations: what you could not test, and what remains inconclusive.

Safe, authorised testing

• Test systems you own or have explicit written authorisation to test. Injection testing against third-party services without permission is unauthorised access, not research.
• Agree the scope in advance: which systems, which behaviours, which environments, and what happens if a serious failure is found mid-test.
• Use staging environments where possible; avoid injecting payloads into production data stores or live tool integrations.
• Keep payloads purposeful. The goal is demonstrating the prohibited behaviour, not maximising harm or producing content you could not justify.
• Report through the agreed channel, and coordinate disclosure timing with the system’s owners.

Checklist: planning a prompt-injection assessment

• Prohibited behaviours defined in writing, with severity criteria agreed.
• All instruction channels inventoried: user input, retrieved documents, tool outputs, previous turns.
• Both direct and indirect injection covered — including content the attacker does not have to be present to plant.
• Multi-turn sequences included, not only single prompts.
• Tool-call records inspected, not only textual responses.
• Baseline behaviour captured for comparison.
• Target version pinned (model version, prompt configuration, retrieval corpus).
• Reproduction attempted for every candidate; reproduction rate recorded.
• Evidence stored: full transcripts, payloads, configuration, timestamps.
• Inconclusive outcomes recorded as inconclusive — not silently dropped or upgraded.

How PROJECTILE approaches injection testing

PROJECTILE TERMINAL is built around the chain this guide describes: define the prohibited behaviour, generate adaptive multi-turn challenges, observe and record responses, examine candidates in context, and verify independently before reporting.

For the wider methodology this fits into, see our guide to LLM red teaming and our approach page. For systems that act through tools, injection interacts with agency — see AI agent security testing.