loss function

advancedwidespread

The loss function shows with a number how much the AI model erred; the goal of training is to make this number as small as possible.

In plain terms

Imagine you are learning to play darts, and after every throw someone tells you how many centimeters from the center of the board the dart landed. This distance is your 'loss': the smaller it is, the better the throw. The loss function does the same for the AI model: after every response, it calculates how far off it was from the correct one.

This is necessary because the machine does not understand that something is "almost good" or "pretty weak." It can only work with numbers, so the error must be converted into a number. From this single number, the training process knows in which direction to adjust the model.

The loss function is not learning itself, but the metric to which learning is aligned. Humans choose it, and thereby decide what the model considers an error. What the metric does not penalize, the model will not avoid.

How it works

The model receives a training example and gives an answer: for instance, it guesses the price of an apartment or the next word in a sentence.
The loss function compares this answer with the correct solution and converts the difference into a single number: small error equals small number, large error equals large number.
The training procedure calculates in which direction to shift the model's internal settings (the weights) slightly so that this number decreases.
The settings are adjusted by a tiny step, then the next batch of examples comes (in practice, the model receives several examples at once). This cycle is repeated hundreds of thousands or millions of times for large models.
Meanwhile, developers monitor the loss curve: if it decreases, the model is learning; if it stalls, the model has either learned what it can or is stuck; if it starts to rise, something needs to be changed.

An everyday example

When your phone keyboard suggests the next word, a model trained to receive a 'penalty' for every wrong guess is at work. The loss function measured how far off the guess was from the actual word typed, and the model adjusted until its suggestions hit more often.

Who is affected

developerIt selects the loss function, thereby determining what the model strives for during training. → Code

entrepreneur, executiveIf you order an AI system, it is worth asking what it was optimized for, because the model learns what is measured, not what you want.

patient, doctorIn a diagnostic model, it also depends on the loss function (and the setting of the decision threshold) whether the system better avoids false alarms or missed diseases. → Health

everyoneBehind every AI tool there is such a benchmark, and it explains why the machine is good at some things and not at others.

A common misconception

MythIf the loss is low, the model is good and reliable.

In realityLow loss only means that the model performs well on the training examples according to the chosen metric. It may still make mistakes in new situations, and the metric might not measure what is important to you.

Limits and risks

The loss function only measures what can be converted into a number; fairness, justice, or usefulness are hard to compress into a single formula.
The model meets the metric, not your intent: if the metric is poorly chosen, the machine will find loopholes, appearing good on paper but useless in practice.
If the loss measured on training examples is pushed too low, it can lead to overfitting: the model memorizes the examples and performs poorly on new data. Therefore, loss is also measured on a separate set of held-out validation data.
The loss value alone means little; it is only comparable if measured on the same task, with the same data, and using the same metric.

Similar, but not the same

accuracy – Accuracy is a post-hoc metric for humans indicating how many answers were correct, while the loss function is an internal measure controlling training that also calculates how bad the wrong answers were.
gradient descent – The loss function indicates the size of the error, while gradient descent is the procedure that reduces it step by step.
reward function – In reinforcement learning, the reward must be increased, and it evaluates the consequence of actions, whereas the loss must be decreased, and it compares the answer with the correct solution.

Related concepts

Broader

machine learning, model training, optimization

Narrower

cross-entropy, mean squared error

On the edge

gradient descent, backpropagation, overfitting, reward function, fine-tuning

A brief history

The idea of measuring and reducing error numerically comes from statistics: Adrien-Marie Legendre published the method of least squares in 1805. In neural network training, it has played a central role since backpropagation was popularized in 1986.

Where things stand

At the end of 2026, virtually every model built on neural networks, including large language models, learns by minimizing a loss function. Research today focuses more on how to choose an objective that better matches human intent.

Status: 2026 November

In the news

Appears in 82 Agent Age stories; the curve shows weekly occurrence in the news magazine.

Election losses could halt up to 30% of large US AI data center projects, analysis saysNov 4, 2026 · Ipar
Little evidence that AI is behind mass job losses or the fall in youth hiringNov 2, 2026 · Work & HR
Anthropic CEO warns AI could erase a large share of entry-level jobs as company heads to Wall StreetNov 2, 2026 · Work & HR
ECB lists AI stock valuations among financial stability risksNov 2, 2026 · Money
OTP: fraudsters take an average of HUF 1.4 million per victim, and AI heightens the threatNov 2, 2026 · Money
Fear of job losses may slow AI adoption at Hungarian companiesOct 30, 2026 · Work & HR
Anthropic files for IPO, set to be first frontier AI lab to listOct 26, 2026 · Agent Age
US state reports from Nevada, Maine and Indiana weigh AI's labour market effectsOct 26, 2026 · Work & HR

Also known as

loss function · loss · loss function · cost function · objective function · loss function · loss · cost function

Sources

Goodfellow, Bengio, Courville: Deep Learning (MIT Press, 2016)

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.