data mixture

advancedwidespread

The data mix shows from which sources and in what proportions the dataset is composed that an AI model learns from.

In plain terms

A language model does not learn from a single large book but from a wide variety of materials: websites, books, source code, scientific articles, dialogues, and even practice exercises written by machines. The data mix is the recipe for this: it specifies how much of each goes into the training material.

It is like goulash. From the same ingredients, a completely different dish results if you add three times as much paprika or half as much meat. Similarly, with models, the ratios determine what they excel at: more code makes a better programmer, more dialogue makes a better conversationalist.

That is why developers do not simply dump everything available. They weigh content consciously: they draw more often from rare but valuable sources (such as Hungarian-language or medical texts) and reduce the volume of low-quality web text.

The data mixture is therefore not the data itself but its composition. Two models can learn from roughly the same sources yet behave differently because they received them in different proportions.

How it works

Developers gather sources and organize them into groups: web text, books, code, dialogue, professional material, machine-generated (synthetic) examples, and various languages.
Each group receives a weight, meaning the percentage of examples from that source during training. The weight of small but important sources is increased, while that of overly large ones is decreased.
During training, the system draws examples according to these ratios. So the same text from a small source may come up several times, while from a huge one only part of it will.
The finished model is measured on test tasks. If it is weak in some area, the proportions are adjusted and they try again, because the right mix is found through experimentation.
The mix can also change between training stages: at the beginning, much general text; at the end, more curated, high-quality, or task-specific examples.

An everyday example

If you ask two chatbots the same question about Hungarian grammar, one may answer smoothly and naturally while the other responds haltingly, like a literal translation. A common reason for this is that one had a significantly higher proportion of high-quality Hungarian text in its training material (though model size and training methods also matter). For partly the same reason, one model may be strikingly good at programming while being weaker at poetry.

Who is affected

everyoneWhether an AI tool is good at something or unreliable depends largely on what and in what proportions it was trained on.

developerWhen training or fine-tuning your own model, choosing the proportions of the data mix matters at least as much as the amount of data. → Code

entrepreneur, executiveWhen choosing a model, it is worth asking how well your specific field and the Hungarian language were represented in the training material.

lawyer, citizenThe composition of training data is a copyright, data protection, and transparency issue, for which the EU AI Act requires providers to submit a summary. → Law

A common misconception

MythThe more data a model receives, the better it becomes; composition is secondary.

In realityQuantity alone is insufficient: a model trained on poorly balanced material will be one-sided. A smaller, well-curated mix often yields better results than a massive, unselected data dump.

Limits and risks

Large companies rarely disclose the exact mix, so outsiders can only infer it from the model's behavior.
There is no single perfect recipe: if the proportion of one area is increased, the model may weaken in another.
Proportions can also introduce bias: in areas that appear little in the mix (small languages, rare specialties, minority perspectives), the model is less certain and makes more errors.
If too much machine-generated text is mixed in, the model's responses may become monotonous and less accurate.

Similar, but not the same

training data – The training data is the material from which the model learns, and the data mix is its composition and proportions.
data augmentation – Data augmentation creates new variations from existing examples, while data mixing decides how much from each source goes into the training.
curriculum-based teaching – The curriculum refers to the order of examples (from easy to hard), while the data mix refers to the proportions of sources.

Related concepts

Broader

training data, machine learning, dataset

Narrower

sampling weighting, Multilingual data mix, task blending

On the edge

synthetic data, pre-training, fine-tuning, data distortion, data curation

A brief history

Proportional mixing of sources is an established practice in machine learning, especially in multi-task learning. Google researchers' 2019 T5 study already compared several mixing strategies, and with the spread of large language models, designing the mix has become a key step in model development.

Where things stand

By the end of 2026, developers of large language models work with deliberately designed data mixes, but the exact recipe remains mostly a trade secret. Since August 2025, the EU AI Act requires providers of general-purpose models to publish a summary of training material content (with a grace period until August 2027 for models already on the market). This summary outlines the main sources but does not reveal exact proportions.

Status: 2026 November

Also known as

data mixture · adatmix · training data mixture · training data composition · data mixture · data mix · training data mixture · dataset mixing

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.