generalization

basicwidespread

Generalization is the AI model's ability to correctly apply what it has learned to new cases it never saw during training.

In plain terms

Think of two students before a math exam. One memorizes the solutions to practice problems, while the other understands the method. If the exam presents the same problem type with new numbers, only the second succeeds. In AI, this second capability is called generalization.

A model is always trained on a finite number of examples, while the real world is infinitely diverse. Therefore, a model is only as good as its performance on unknown cases. Good results on training examples alone tell little.

The failure of generalization is called overfitting in technical terms. In this case, the model memorizes tiny details and random peculiarities of the training data as if by rote, and makes mistakes in new situations.

How it works

Developers split the data: the model trains on the larger portion, while a smaller part is set aside for testing and remains invisible during training. Often, a third segment is separated to verify adjustments made during the training process.
During training, the model searches for patterns and regularities in the examples and adjusts its internal settings accordingly.
After training, performance is measured on held-out, unknown examples. The difference between results on learned data and new data shows how well the model generalizes.
If a model performs excellently on the data it was trained on but poorly on new data, it is overfitted. In such cases, it is improved by using more and more diverse data, techniques that curb the model's excessive adaptation (known as regularization), simpler models, or by stopping training earlier.
Even after deployment, the model must be monitored because the world changes, and the performance of a previously well-generalizing model may deteriorate over time.

An everyday example

Your phone's photo app finds images of your own dog when searching for "dog," even though it never saw your dog during training. It learned what makes a dog a dog from millions of photos of other dogs and applies this to new images.

Who is affected

everyoneEvery AI tool you use is as reliable in your specific situation as it is good at generalizing from learned examples.

developerThe model's value is determined by its performance on unknown data, so testing and data splitting must be carefully planned. → Code

patient, doctorA diagnostic model trained on a hospital's machines and patients may perform worse elsewhere, with different devices and patient populations. → Health

entrepreneur, executiveAn AI solution that performs well in a demo may behave differently on your company's own data, so it is worth testing on your own cases before purchasing.

A common misconception

MythIf a model achieves excellent results on tests, it will perform equally well in real life.

In realityThe test only shows how the model performs on cases similar to the test tasks. If the real situation differs, performance can drop significantly.

Limits and risks

The model typically generalizes well only to situations similar to its training data; it can become unreliable in very unusual or rare cases.
The model can give a confident answer even when operating in unfamiliar territory, so the error is not always noticed.
Results from public leaderboards can be skewed because some test tasks may have been included in the training data, and the model may have simply memorized them.
If a group or situation was missing from the training data, the model generalizes poorly to it, which can lead to unfair decisions.

Similar, but not the same

overfitting – Overfitting is a failure of generalization: the model memorizes the training examples instead of learning the underlying rule.
memorization – Memorization is the reproduction of seen examples; generalization is applying rules derived from them to new cases.
transfer learning – Transfer learning is a method where knowledge gained from one task is applied to another; generalization is the ability of the model to handle new cases.

Related concepts

Broader

machine learning, model training

Narrower

out-of-distribution generalization

On the edge

overfitting, underfitting, training data, tesztadat, benchmark, zero-shot learning

A brief history

The mathematical theory of generalization was founded by Vladimir Vapnik and Alexey Chervonenkis in their 1971 work, and Leslie Valiant provided a formal framework in 1984 for what it means to learn reliably from examples. In 2016, a highly cited study showed that deep neural networks can even memorize random labels yet still generalize well, a phenomenon the classical theory does not explain satisfactorily.

Where things stand

Generalization is a fundamental concept in machine learning and the central metric for evaluating every model. By the end of 2026, it remains an open research question to what extent large language models and agents truly generalize to new situations versus merely repeating learned patterns.

Status: 2026 November

Also known as

generalization · generalization · Generalization ability · generalization capability · generalization · generalisation

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.