jutalommodell (RM)

in English: reward model

advancedwidespread

An auxiliary model that scores how good an AI response is, so humans do not have to rate every single response during training.

In plain terms

Imagine that a master chef had to taste every single dish a cooking apprentice makes. That is impossible, so the master trains a taster who knows their palate well enough to score on their behalf. The reward model is this taster: a separate AI model that learned from human judgments what we consider a good response.

It is needed because a "good answer" cannot be described by a formula. In a chess game, it is clear who won, but whether an answer is helpful, polite, and accurate is very difficult to codify into rules. People, however, can point out which of two answers is better, and from this the reward model learns the pattern.

Once created, the reward model scores orders of magnitude faster and cheaper than humans. The large language model learns from these scores which responses to generate more often and which to avoid.

Important distinction: the reward model is not the chatbot you are talking to, but the 'jury' working in the background that shaped its behavior during training.

How it works

Developers ask the language model to generate multiple answers to the same question.
Human evaluators compare responses and mark which one they consider better, or rank them.
A separate model is trained on these judgments, which assigns a single number to any question–answer pair: the higher the number, the more likely it is to be liked by people.
The language model generates new responses, the reward model scores them, and reinforcement learning pushes the language model toward high-scoring behavior.
Meanwhile, developers monitor whether the model starts gaming the scorer, and if necessary, they improve the reward model with fresh human evaluations.

An everyday example

When a chatbot responds politely and in a structured way, rather than tossing out the first sentence that comes to mind, it is partly the trace of a reward model's work. During training, it received high scores for responses in this style, so it now gives them to you.

Who is affected

everyoneChatbots' tone, caution, and helpfulness are largely determined by what the reward model rewarded during their training.

developerIf you tune your own model or train an agent, the quality of the reward model determines whether the system learns the true objective or merely chases the score. → Code

entrepreneur, executiveThe behavior of an AI product is underpinned by value choices: it is worth asking the provider whose judgments the system learned from.

employeeThe evaluators and labelers required to train reward models are humans, creating a new, often underpaid job category worldwide. → Work & HR

A common misconception

MythThe reward model knows what is true and correct, so anything it scores highly is certainly a good answer.

In realityIt only estimates what the people who trained it would have scored highly. If they preferred confident, appealing answers, the reward model also rewards these, even if they are incorrect.

Limits and risks

A trained model may learn to game the scorer: it gives answers that receive high scores but are not actually better (this is called reward hacking).
The reward model inherits human evaluators' biases, cultural preferences, and errors, and may even amplify them.
It tends to reward flattering responses that agree with the user, because these please people in the short term.
In situations where evaluators have not seen similar cases, the score may be unreliable.

Similar, but not the same

reward function – The reward function is the broader term: any rule that specifies how much reward is given for a behavior. Traditionally, humans write this manually (for example, points in a game). The reward model is its learned variant: it learns from data and human judgments what to reward.
loss function – A loss function is a mathematical formula that measures how much the model's output deviates from the expected result, and training directly reduces this. In contrast, a reward model is itself a trained model (it is also trained with a loss function) and provides an evaluation when there is no single correct answer.
judge model (LLM as a judge) – A critic model typically evaluates the finished system and also explains its verdict in text, whereas a reward model assigns a score during training that directly shapes the model. The boundary is not sharp: a critic model's score is also used for training.

Related concepts

Broader

reinforcement learning, machine learning, model training

Narrower

preferenciamodell, folyamatjutalom-modell

On the edge

reinforcement learning from human feedback, fine-tuning, reward hacking, AI alignment, data labeling

A brief history

The concept of reward learning from human feedback is older, but it gained prominence in deep learning in 2017 with Paul Christiano and colleagues' paper 'Deep reinforcement learning from human preferences', which demonstrated it on games and simulated robots. On language models, OpenAI tested it for text summarization in 2020, and the 2022 InstructGPT paper made it widely known. This method underpinned the training of ChatGPT.

Where things stand

As of late 2026, the reward model remains an established tool in training large language models and agents, but not the only one. Some methods (such as DPO) learn directly from human preferences without a separate reward model. For verifiable tasks (code, mathematics), automatic checking often supplies the reward. A main research question is how to prevent the scorer from being gamed and sycophantic behavior from being rewarded.

Status: 2026 November

Also known as

jutalommodell · jutalom modell · reward model · reward model · RM · preferenciamodell · preference model · learned reward model

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.