GRPO

in English: group relative policy optimization

professionalspreading

A training method where the AI model provides multiple answers to a task and learns from those that perform better than the group average.

In plain terms

Imagine a teacher having a student solve the same task eight times, grading each solution, then showing which was better or worse compared to the average of the eight. It is not the score itself that matters, but the deviation from the average. The student reinforces the thought process leading to better results and discards the weaker ones. GRPO does exactly this with a language model.

The method belongs to the family of reinforcement learning: the model does not copy ready-made sample answers but tries, and receives a score for its attempts. It is mainly used to help the model think step by step, for example on mathematical problems or in programming.

In previous methods, a separate "critic" model had to be trained to predict how good a situation was. GRPO eliminates this: the reference point is the group's average itself. This makes training simpler and requires less memory and computation.

How it works

The model generates multiple different answers to the same question; this is the group.
A scorer evaluates every answer, for example checking whether the final result is correct or if the written program runs.
The system calculates the group's average and standard deviation, and for each response checks how much better or worse it is than the average.
The model's internal settings are adjusted so that above-average responses become more likely and below-average ones rarer.
A brake ensures that the model does not change too much in one step and does not stray far from its initial state.

An everyday example

A 'thinking' chat model first derives the solution at length for a math problem and only then gives the answer. In open models, such as DeepSeek-R1, this capability was reinforced with GRPO: during training, the model solved many tasks in multiple ways, and derivations leading to the correct final result became more probable within it. Manufacturers of closed models mostly do not disclose exactly what method they use.

Who is affected

developerIf you want to fine-tune an open model for your own task, GRPO is one of the most accessible reinforcement methods, supported by several open tools. → Code

entrepreneur, executiveThe method made reinforcement learning for reasoning models simpler and cheaper, making custom models more realistic for smaller teams.

parent, student, teacherSuch step-by-step, problem-solving AI assistants are often based on this type of training. It is worth knowing that in these cases, the model is typically rewarded for the correct final result, not for the good explanation, so the reasoning can be flawed or misleading even if the final result is good. → Education & Family

A common misconception

MythGRPO is a new type of AI model that can be downloaded and used.

In realityGRPO is not a model, but a training procedure: it describes how to improve an existing model through trials and scoring.

Limits and risks

It works well only where answers can be automatically and reliably scored (by a program, a validation rule, or a critic model); without a reliable scorer, the model does not know what to align with.
The model is prone to gaming the scorer: it can learn how to get a high score without actually solving the task well.
Many answers must be generated for every question, so training also requires significant computation.
If every response from the group is equally good or equally bad, there is no difference to learn from, so tasks that are too easy or too hard are of little value.

Similar, but not the same

PPO (proximal policy optimization) – PPO trains a separate value estimation model as a reference point, while GRPO uses the average of the response group instead.
supervised fine-tuning – In supervised fine-tuning, the model imitates ready-made sample answers, while in GRPO it learns from its own attempts and their scores.
RLHF (reinforcement learning from human feedback) – RLHF specifies where the reward comes from (human judgments), while GRPO specifies how the model is updated based on the reward.

Related concepts

Broader

reinforcement learning, fine-tuning, machine learning

Narrower

–

On the edge

PPO (proximal policy optimization), RLHF, reward function, reasoning model, chain of thought

A brief history

The method was described by DeepSeek researchers with university co-authors in February 2024 in their DeepSeekMath study. It became widely known in January 2025 when presented as a key element in training the DeepSeek-R1 reasoning model.

Where things stand

By the end of 2026, GRPO is a standard starting point for reinforcement learning in open reasoning models, supported by popular open-source training tools. Meanwhile, researchers keep publishing improved versions that address its stability and bias problems.

Status: 2026 November

Also known as

GRPO · group relative policy optimization · Group Relative Policy Optimization · group-relative policy optimization · group relative policy optimization · group-relative policy optimization · GRPO training · GRPO algoritmus

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.