data poisoning

advancedspreading

Data poisoning is an attack where someone intentionally smuggles false or manipulated data into an AI's training material so that the system later operates incorrectly or in the attacker's favor.

In plain terms

Imagine someone secretly altering a few recipes in a culinary school textbook. Students learn them in good faith, and years later still cook the dish poorly because the error is embedded in their knowledge. Data poisoning does the same to artificial intelligence: it does not break the finished system but corrupts what it learns from.

The attacker's goal can vary. They may want to generally degrade the model's quality. They may want to distort its answers on a specific topic. They may also embed a hidden 'backdoor', meaning the model behaves normally until a certain password or pattern appears, and then does what the attacker wants.

It is hard to defend against because today's large models learn from vast amounts of text and images collected from the internet, which no human ever reviews individually. Anyone who can put content on the public web can theoretically inject something into the training material.

How it works

The attacker identifies where the model collects data: public websites, forums, open code repositories, downloadable datasets, or user feedback.
It places poisoned content: false claims, mislabeled images, or texts where a rare term is always linked to the same harmful behavior.
The developer collects the data, and during training the model learns the poisoned samples just like the real ones.
The finished model performs well on standard tests, so the flaw stays hidden.
When the trigger condition occurs (a specific question, keyword, or image fragment), the model responds or acts as trained by the attacker.

An everyday example

An online store's recommendation system learns from customer reviews. If a competitor uploads thousands of fake reviews, the system learns to rank good products lower and push weaker ones to the top. You only see that the recommendations are strange.

Who is affected

everyoneIf a chatbot or search engine has learned from poisoned data, it can confidently give false or biased answers, and you cannot see this from the outside.

developerDownloaded open datasets, pre-trained models, and code repositories may all contain poisoned content, so the origin of the data must be verified just like the code. → Code

entrepreneur, executiveIf your company trains or tunes a model on its own data, manipulated data can lead to incorrect business decisions and liability issues.

patient, doctorIn diagnostic-support systems, even a few deliberately mislabeled scans can lead to incorrect findings. → Health

A common misconception

MythA large model can only be poisoned if a significant portion of the training data is falsified.

In realityResearch suggests surprisingly few samples may be enough: in a 2025 study (Anthropic, the UK AI Security Institute, and the Alan Turing Institute), roughly 250 poisoned documents were sufficient to embed a simple backdoor, and this number did not depend on model size for the tested models of up to 13 billion parameters. It is not yet known whether this holds true for the largest models and more complex attacks.

Limits and risks

Poisoning is difficult to detect afterward because the model may perform flawlessly on standard tests.
Poisoned knowledge already learned cannot be simply erased; often only retraining on cleaned data helps.
Perfect protection does not exist: data filtering and source verification reduce risk but do not eliminate it.
Not every faulty AI response stems from an attack; most errors are simply due to weak or incomplete training data.

Similar, but not the same

prompt injection – Prompt injection hijacks a finished model during use with a smuggled instruction, while data poisoning corrupts the model already during training.
bias – Bias is usually unintentional, stemming from one-sided data, whereas data poisoning always involves deliberate interference.
hallucination – In hallucination, the model invents falsehoods on its own; in poisoning, someone deliberately trained it to do so.

Related concepts

Broader

AI safety, cybersecurity, adversarial machine learning

Narrower

backdoor attack, label poisoning

On the edge

training data, prompt injection, bias, supply chain attack, fine-tuning, model poisoning

A brief history

Researchers have been studying the vulnerability of machine learning since the mid-2000s. One of the first and most frequently cited studies is the 2012 work by Battista Biggio, Blaine Nelson, and Pavel Laskov, which showed that a learning algorithm can be deliberately sabotaged by injecting patterns into the training data. The general public encountered the phenomenon in 2016 when Microsoft's Tay chatbot, which learned from conversations with users, was trained to produce offensive text in less than a day and had to be shut down.

Where things stand

At the end of 2026, data poisoning is already a recognized security risk: the EU AI Act mentions it by name, and NIST has also classified it among attack types. No proven, reliable defense exists yet; developers mainly rely on verifying the provenance of data and filtering it.

Status: 2026 November

Also known as

data poisoning · data poisoning · training data poisoning · poisoning attack · poisoning attack · training data poisoning · AI poisoning

Sources

Biggio, Nelson, Laskov: Poisoning Attacks against Support Vector Machines (2012)

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.