scaling laws

advancedwidespread

Empirical correlations about how much an AI model's performance improves when increasing its size, the amount of training data, and computing capacity.

In plain terms

Scaling laws are neither legal statutes nor laws of nature, but observed regularities. They describe how, when a language model is built larger, trained on more text and given more computing time, its error rate falls at a surprisingly predictable pace.

It is like an athlete's training log: if you see how much each doubled training volume improved your time so far, you can cautiously estimate the next step. The same applies here: every further improvement requires more input; performance does not grow linearly, and the effort usually must be multiplied for each new step.

This is why they are important: development companies partly decide based on these whether it is worth spending billions on a larger model and the required data center. Scaling laws also explain why the AI race has partly become a race for chips, electricity, and data centers.

How is this different from a new idea or invention? The question here is not how to devise a smarter method, but how much it brings if we simply use more of the same method.

How it works

Researchers train many variations from the same model type: smaller and larger, with less and more data, and with shorter and longer training.
In each case, they measure how much the model errs, for example, how accurately it predicts the next word in a text.
They plot the results on a graph and see that the points fit a regular curve: the error decreases according to a so-called power law. This means that every tenfold increase in investment reduces the error by roughly the same proportion, but it cannot go below a certain level.
By extending the curve, they estimate what a much larger, not-yet-built model could achieve and how much its training would cost.
Ratios are also calculated: given a budget, what model size and data volume should be chosen to maximize the return on investment.

An everyday example

You read in the news that an AI company is building a new data center for tens of billions of dollars, consuming as much electricity as a large city. Behind the decision are scaling laws: the company expects that more computing capacity will result in a predictably better model, which will then appear in your chat application.

Who is affected

developerIt helps estimate whether a smaller, cheaper model is sufficient for a task or if paying for a larger one is truly worth it. → Code

entrepreneur, executiveThis explains why model capabilities and prices change rapidly and why it is not advisable to build a long-term strategy on a single model version.

bank customer, investorInvestment expectations around chipmakers, cloud providers, and energy companies largely rely on the assumption that scaling will continue to be profitable. → Money

everyoneThis partly explains why chatbots and agents are becoming noticeably better year after year, while AI's electricity and water demand is simultaneously increasing.

A common misconception

MythScaling laws guarantee that larger models will always be smarter in every way, indefinitely.

In realityThese are curves fitted to past measurements, not laws of nature: no one knows for sure how long they remain valid. Moreover, an improvement in a well-measured error count does not automatically mean the model will be more reliable or useful for your task.

Limits and risks

Curves are reliable only as long as measurements continue; predictions beyond that are estimates, not facts.
Improvement is becoming increasingly expensive: each new step requires many times more chips, electricity, and money than the previous one, posing economic and environmental limits.
High-quality training data is finite. Original laws assumed there would always be enough new data; if data must be reused or replaced with lower-quality data, improvement slows.
In the news, the term is often used as a business argument, so it is worth examining what interest the party citing it has in continued scaling.

Similar, but not the same

Moore's Law – Moore's Law describes the observation that the number of transistors on chips roughly doubles every two years. Scaling laws, on the other hand, concern how much improvement in AI model performance the invested resources bring.
scalability – Scalability refers to whether a system can handle more users or load, while scaling laws describe the growth in model quality.
benchmark – A benchmark measures a single model's performance on a specific set of tasks, while a scaling law is a trend drawn from many measurements, allowing for predictions.

Related concepts

Broader

machine learning, deep learning, artificial intelligence

Narrower

Chinchilla scaling, inference-time scaling, compute-optimal training

On the edge

large language model, parameter, computing capacity, training data, Moore's Law

A brief history

Similar regularities were described by Baidu researchers in 2017, but the concept became widely known in 2020 when OpenAI researchers (Jared Kaplan and colleagues) published their study on the scaling laws of language models. In 2022, a DeepMind study known as 'Chinchilla' (Jordan Hoffmann and colleagues) refined the ratios: it showed that previous models were trained on too little data relative to their size. Since then, companies often train smaller models on far more data than the Chinchilla ratio, because this makes the model's use cheaper.

Where things stand

At the end of 2026, scaling laws are the established basis for the investment decisions of major AI developers, but there is lively debate about how much returns are diminishing. Attention is also increasingly turning to how much computation, or 'thinking time,' a model gets while answering.

Status: 2026 November

Also known as

scaling laws · scaling law · scaling laws · neural scaling laws · neural scaling laws · scaling laws · Chinchilla scaling · AI scaling

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.