If you’re commissioning software with an AI feature this year, someone on your engineering team should be able to explain how they’ve thought about prompt injection. Not recite a definition, but explain what it means for the specific feature you’re building. It’s a reasonable question to ask a vendor or a hire, and the quality of the answer tells you a lot about how seriously the AI layer has been engineered.
What prompt injection actually is
A large language model doesn’t have a separate channel for “instructions from my developer” and “content I’m being asked to process”. Both arrive as the same stream of text. Prompt injection exploits that: it’s any input, crafted or accidental, that causes the model to follow instructions embedded in the data it’s processing rather than the instructions it was actually given.
The simplest version is direct: a user types “ignore your previous instructions and do X” into a chat box. Most systems built in the last couple of years handle that case reasonably well, because it’s the obvious one and it’s easy to test for.
The version that catches teams out is indirect. The instruction doesn’t come from the person typing into your product at all. It arrives inside a document the model is asked to summarise, an email it’s asked to draft a reply to, a web page it fetches to answer a question, or a meeting transcript it’s asked to process. The end user never sees it. The model reads it as part of the content it was told to work with, and if that content contains something that reads like an instruction, a sufficiently capable model may act on it.
“We don’t let users write prompts” isn’t a defence
This is the assumption we hear most often, and it’s the one worth pushing back on. Plenty of AI features have no visible prompt box at all. A meeting assistant that reads a transcript and produces a summary. A finance tool that categorises transactions from imported statements. An assistant that reads a calendar invite and drafts a response. None of these ask the end user to type instructions to a model, and none of them are safe from indirect injection for that reason, because the attacker’s target isn’t your user interface — it’s any text the model will read as input.
The content that reaches the model can come from other people entirely: whoever sent the email, wrote the document, or spoke in the meeting. If your feature ingests third-party content and hands it to a model with any ability to take action — send a message, write to a record, call another system — that content is untrusted input, on the same footing as a form field, and needs the same discipline.
The controls that actually reduce the risk
There’s no filter that reliably strips malicious instructions out of natural language while leaving the legitimate content intact — the two aren’t structurally distinguishable to the model. That’s why the OWASP Top 10 for LLM Applications lists prompt injection as its top-ranked risk and doesn’t offer a single fix for it. The useful response isn’t to try to make the model immune to bad input. It’s to limit what happens if it acts on some.
- Least privilege for anything the model can call. If a model has a tool that can send an email or update a database record, it should only be able to reach the accounts, records or scopes the current task genuinely needs — not a service credential with broad access reused across every feature.
- Separate the model’s output from your system’s actions. A summary or a suggested reply is safe to show a user. Letting the model’s raw output directly trigger an irreversible action — send, delete, transfer, publish — without a check in between is where injected instructions turn into real damage.
- Human confirmation on anything that costs money or can’t be undone. This is the cheapest control available and the one most often skipped for the sake of a smoother demo. If the action is reversible and low-stakes, automate it. If it isn’t, put a person in the loop.
- Treat model output as untrusted downstream too. If the model’s response gets rendered as HTML, inserted into another prompt, or passed to another system, it can carry an injection forward. Escape and validate it the same way you would any other user-supplied string.
- Log what the model was asked to do and what it did. When something goes wrong, you need to be able to see the input that caused it. Without logging the actual prompts and tool calls, debugging an injection incident is guesswork.
Where this bites small teams specifically
Most of the damage we see in write-ups of real incidents doesn’t come from a sophisticated attack. It comes from a feature that was scoped as “add an AI summary” and shipped with a service account that had far more access than the summary feature needed, because that credential already existed and reusing it was faster than provisioning a narrower one. A small team under time pressure will make that trade-off without necessarily framing it as a security decision, because in the moment it doesn’t feel like one.
The fix isn’t a bigger security budget. It’s asking, before the feature ships, exactly what the model is allowed to touch, and whether that scope matches what the feature actually needs to do — not what was convenient to wire up.
Questions worth asking before you commission an AI feature
- Does this feature process content written by anyone other than the end user — emails, documents, transcripts, web pages? If so, that content is untrusted input.
- Can the model trigger an action, or does it only produce text a human reads? If it can act, which actions are reversible and which aren’t?
- What credentials or API scopes does the model’s tooling actually hold, and is that the minimum the feature needs?
- Is there a human checkpoint before any action that costs money, sends something externally, or can’t be undone?
- If this went wrong tomorrow, is there a log that would show you what the model was told and what it did?
If your team can answer all five without hesitating, the AI feature has been engineered with this in mind. If the answers are vague, that’s the gap to close before launch, not after.