Prevention
Preventing data poisoning: the controls that matter
No control removes the risk, but a few reduce it a great deal: know where data comes from, version it, vet suppliers and keep unverified sources away from training.
Last reviewed against Anthropic, UK AI Security Institute and arXiv. Sources
The controls
What to put in place
Work down the list in order, as each one builds on the one before.
-
1
Track the origin and transformation of every dataset, and verify its legitimacy at each stage of development [4]. -
2
Version datasets so that manipulation can be detected [4]. -
3
Vet data vendors carefully [4]. -
4
Sandbox access to unverified data sources and use anomaly detection to filter adversarial data [4]. -
5
Restrict infrastructure so models cannot reach data sources they should not use [4].
Tooling
Records you can keep
OWASP gives examples of record keeping: CycloneDX or an ML bill of materials for provenance, and DVC for versioning datasets [4]. The aim is the same whichever tool you use: a record of what went into each model.
Design choices
Reduce exposure by design
OWASP also suggests fine-tuning on datasets specific to each use case, and storing user-supplied information in a vector database so it can be adjusted without retraining the whole model [4]. Both narrow what a poisoned input can affect.
Also consider:
- Limiting who can add to training and retrieval stores
- Keeping production and experimental fine-tuning apart
- Treating downloaded model files as software, given OWASP's warning about malicious payloads in shared models [4]
Keep reading
Read next
Detection
How to look for signs of poisoned data: output checks, adversarial testing, monitoring and why benchmarks alone are not enough.
Read the guide
Guidance
Where UK, US and industry guidance covers data poisoning: the NCSC guidelines and principles, NIST's taxonomy and OWASP's LLM risk list.
Read the guide
What to do
A practical order of work for UK organisations to reduce data poisoning risk in AI systems: inventory, provenance, suppliers and testing.
Read the guideHow CyPro can help
Need a hand planning for this?
Understanding the change is one job. Working out what it means for your systems, your suppliers and your roadmap is several more. CyPro's consultants can take any of them on.
Fractional CISO for the board
A senior security lead who advises the board on what to fix first, what to accept and what to tell customers, and owns the decision record.
More on cypro.co.uk
Hardening and segmentation
Our consultants put the controls in place that limit the damage when something is missed: firewall rules, network isolation and tighter admin access.
More on cypro.co.uk
Penetration testing
A focused test of your systems and the controls around them, so you know they hold before an attacker finds out for you.
More on cypro.co.uk
Vulnerability scanning
Find every internet-facing system you run, including the forgotten ones, and see which are still on a vulnerable build.
See the scanning service
24/7 security monitoring
Our SOC analysts watch your systems day and night for the signs of an attack in progress: unexpected processes, new outbound connections, logins that do not fit.
More on cypro.co.uk
Incident response
If you may already be compromised, our responders contain it, preserve the evidence, find out what was touched and get you running again safely.
More on cypro.co.uk
Plan ahead
Want a plan for your organisation?
Defending against data poisoning starts with knowing which AI systems you run, where their data comes from and who owns the risk. CyPro can help set the governance, risk assessment and controls around that.