Prompt injection

Prompt injection: controls that reduce the risk

A complete fix for prompt injection may never arrive, so the working aim is to make attacks rarer and less damaging. These controls come from the NCSC and OWASP.

Last reviewed against NCSC, Department for Science, Innovation and Technology and OWASP GenAI Security Project. Sources

Dark blue arched badge with a large green and blue tick across it

The problem

Why there is no patch

In the NCSC's analysis, today's LLMs draw no enforced line between the instructions in a prompt and the data placed alongside them [1]. Parameterised queries cured SQL injection, but the NCSC thinks nothing equivalent is likely for prompt injection [1]. It suggests picturing the model as an "inherently confusable deputy" [1].

As reported by The Record, an NCSC technical director put it this way: prompt injection will stay a residual risk, and no product or appliance removes it [11].

Controls

What OWASP and the NCSC advise

Two server towers, one on fire, linked by a cycle of arrows to a laptop

Constrain behaviour

Tell the model its role, scope and limits, and check expected output formats with ordinary code rather than with the model [9].

Blue server connected to a laptop, a monitor, a folder and a green shield

Separate untrusted content

OWASP advises segregating and labelling external content [9]. The NCSC gives wrapping external content in XML tags as an example of making attacks harder [1].

Tall blue server rack with glowing status lights

Filter, carefully

OWASP lists input and output filtering [9]. The NCSC cautions against leaning on deny-lists, because attackers can reword their input [1].

Cloud with a combination lock dial at its centre

Limit privileges

Keep sensitive functions in application code and give the model the least privilege it needs [9].

Factory buildings linked through a hub to a phone showing a fingerprint and shield

Test

Run regular adversarial testing and treat the model as an untrusted user [9].

Monitoring

See the attempts

The NCSC recommends keeping logs of what goes into the model, what comes out, which tools it used and which APIs it called. Calls that fail may show an attacker honing an attempt [1]. Review those logs on a schedule, and send alerts to someone with the authority to switch a tool off.

When to choose another approach

For a system whose security needs leave no room for what is left over, the NCSC says an LLM may not be the right fit [1].

Megaphone beside a padlock, shield, firewall, cloud and password field

Awareness

Developers and risk owners

The first NCSC recommendation is awareness: developers and risk owners should understand that prompt injection is a residual risk that has to be managed through design, build and operation [1]. Teams used to treating it like SQL injection need that message most.

How CyPro can help

Need a hand planning for this?

Understanding the change is one job. Working out what it means for your systems, your suppliers and your roadmap is several more. CyPro's consultants can take any of them on.

Security lead advising a board

Fractional CISO for the board

A senior security lead who advises the board on what to fix first, what to accept and what to tell customers, and owns the decision record.

More on cypro.co.uk
Cyber security consultants planning hardening work

Hardening and segmentation

Our consultants put the controls in place that limit the damage when something is missed: firewall rules, network isolation and tighter admin access.

More on cypro.co.uk
Penetration tester probing an application for weaknesses

Penetration testing

A focused test of your systems and the controls around them, so you know they hold before an attacker finds out for you.

More on cypro.co.uk
Scanner checking internet-facing systems for weaknesses

Vulnerability scanning

Find every internet-facing system you run, including the forgotten ones, and see which are still on a vulnerable build.

See the scanning service
Security analysts monitoring systems around the clock

24/7 security monitoring

Our SOC analysts watch your systems day and night for the signs of an attack in progress: unexpected processes, new outbound connections, logins that do not fit.

More on cypro.co.uk
Incident responders containing an attack

Incident response

If you may already be compromised, our responders contain it, preserve the evidence, find out what was touched and get you running again safely.

More on cypro.co.uk
Rocket launching from a cloud

Plan ahead

Want a plan for your organisation?

AI security starts with knowing which tools are in use, who owns them and what they can reach. CyPro can help set the governance, accountability and controls around that.