tesztadatok

in English: test data

basicwidespread

Test data consists of set-aside examples that the AI model never saw during training, on which its actual performance is finally measured.

In plain terms

Imagine a student practicing on textbook exercises, but in the exam receiving tasks they have never seen before. Only then does it become clear whether they understood the material or simply memorized the solutions. Test data are the exam questions of AI models.

Developers set aside part of the available data at the very beginning of the work. The model learns from the rest, and no one touches the set-aside part until the final measurement. The result achieved here shows what to expect when the model encounters new, unknown cases in real life.

This differs from training data, from which the model learns, and validation data, on which it is tuned during development. Test data serves one purpose only: a fair, post-hoc exam.

How it works

Developers split the collected data into several parts: the largest is used for training, a smaller one for tuning, and a further one for testing.
The model learns exclusively from training data; test data is kept isolated during this time.
Once the model is ready, it receives the questions from the test data, but not the correct answers.
Model answers are compared with correct solutions, and metrics are calculated from this, such as accuracy, meaning the proportion of correct answers.
If the result is much worse than on the training data, the model likely memorized and did not acquire generalizable knowledge.

An everyday example

A company advertises that its spam filter is 99 percent accurate. This figure was generated on test data: thousands of emails that the filter did not see during training. Whether it works as well in your inbox depends on how much your emails resemble those in the test.

Who is affected

everyoneAccuracy figures in AI product ads and news are almost always results measured on test data, so it is worth knowing what they mean.

entrepreneur, executiveBefore purchasing an AI solution, it is worth testing it on a test set compiled from your own data, because the manufacturer's test does not concern your customers.

developerIf test data leaks into training by any means, the measured result will be falsely good, and the error will only surface in live use. → Code

patient, doctorThe reliability of a diagnostic AI depends on whether it was tested on data from patients similar to those appearing in the clinic. → Health

A common misconception

MythIf a model scores 95 percent on a test, it will work well 95 percent of the time in my case.

In realityTest results are valid only for situations resembling the test data. If your data differs (different language, different customer base, different image quality), performance may be significantly weaker.

Limits and risks

Test data only measures what is included: if rare or unusual cases are missing from it, you learn nothing about them.
Public test questions can end up on the internet, and from there in the training material of large language models, so the model is tested on tasks it has seen, and the result is deceptively good.
If developers test frequently on the same benchmark and always adjust to it, the model gradually adapts to the test, not reality.
The world is changing, so test data compiled a few years ago may no longer reflect real-world use.

Similar, but not the same

training data – The model learns from training data, while test data are used solely to evaluate the finished model.
validation data – Validation data is used to tune the model and choose between variants during development, while test data serves for the final, untouched measurement.
benchmark – A benchmark is a standardized, mostly public set of tasks used to compare multiple models, although the test portion is sometimes kept secret. However, every development has test data, even if it is internal and confidential.

Related concepts

Broader

adathalmaz, machine learning, model evaluation

Narrower

hidden test set

On the edge

training data, validation data, benchmark, overfitting, data breach, generalization

A brief history

The principle that a model must be tested on held-out data not used for training came into machine learning from statistics. It became widely known through public datasets such as MNIST, a collection of handwritten digits compiled in the 1990s, which has a separate training set (60,000 images) and test set (10,000 images).

Where things stand

By the end of 2026, the use of test data will be a fundamental requirement for all serious AI development. The EU AI Act also sets quality requirements for training, validation, and test data for high-risk systems (Article 10); these obligations will become applicable gradually according to the regulation's subsequently amended schedule. The greatest challenge is evaluating language models and agents, as public tests quickly leak into training data.

Status: 2026 November

In the news

Appears in 1 Agent Age stories; the curve shows weekly occurrence in the news magazine.

Claude Code deleted customer data and notes while given a clean-up taskNov 4, 2026 · Code

Also known as

tesztadat · tesztadatok · test set · teszthalmaz · testing data · test data · test set · testing data

Sources

Wikipedia: Training, validation, and test data sets

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.