learning rate (LR)

advancedwidespread

The learning rate determines how large the steps are by which an AI model adjusts itself during training as it learns from its mistakes.

In plain terms

Imagine trying to descend a mountain into a valley through thick fog. You can only feel the slope of the ground under your feet, so you always step in that direction. The learning rate is how big a step you take at once.

If you take steps that are too large, you jump over the valley and end up on the opposite side, possibly higher than where you started. If you take steps that are too small, you progress safely, but the journey takes days, and you may get stuck in a small depression that you mistake for a valley.

Model training works the same way: the "valley" is the state where the model makes the fewest mistakes. The learning rate is not part of the model's knowledge but a training setting chosen by developers, on which the success of training often depends.

How it works

The model receives a set of training examples, answers them, and the system calculates how much it erred.
A mathematical procedure shows which direction to shift the model's internal numbers (the weights) to reduce the error.
The learning rate is a small number (for example, 0.001). Simply put: this multiplies the correction, determining whether the model changes significantly or only slightly in one step.
This cycle repeats thousands or millions of times, and the model gradually improves.
In practice, the rate is rarely constant. It usually starts at a low value, quickly rises to the desired level, and then gradually decreases towards the end of training so the model can finely 'tune in' to the correct solution.

An everyday example

A company wants to fine-tune a language model on its own customer service emails. If the developer sets the learning rate too high, the model may 'forget' its previous general knowledge and give incoherent answers. If it is set too low, the expensive computation runs just the same, but the model hardly changes, so it has to be run again.

Who is affected

developerThis is one of the most important settings to choose correctly when training or fine-tuning a model. With a wrong value, training will be slow or may fail. → Code

entrepreneur, executivePoorly configured training wastes compute time and money, so it is worth understanding why the team requests more trial runs.

parent, student, teacherA vivid example of how a machine learns from mistakes in small steps, making it well-suited for explaining how AI works. → Education & Family

A common misconception

MythThe higher the learning rate, the faster and better the model learns.

In realityWith too high a rate, the model rushes: it jumps back and forth around the good solution or completely breaks down during training. The goal is not the largest, but the appropriate step size.

Limits and risks

There is no universally valid good value: it depends on the model, data, and task, so it usually requires trial and error to find.
An excessively high rate leads to unstable training and, during fine-tuning, can destroy the model's existing knowledge.
A rate that is too low means slow and expensive training, and the model may get stuck at a mediocre solution.
The learning rate only matters during training; once the model is ready for use (such as when you chat with it), it no longer plays a role.

Similar, but not the same

learning curve – The learning curve shows how the model's performance improves during training, while the learning rate is a setting that influences this improvement.
temperature – Temperature controls the variability of a trained model's responses during use, while the learning rate sets the step size during training.
epoch (training cycle) – An epoch means how many times the model runs through the entire training dataset, while the learning rate indicates how much it changes in each step.

Related concepts

Broader

Hyperparameter, machine learning, model training

Narrower

learning rate scheduling, warmup, adaptive learning rate

On the edge

gradient descent, backpropagation, loss function, fine-tuning, overfitting

A brief history

The concept of step size originates from mathematical optimization. The method of stepping in the direction of the slope (gradient descent) was described by Augustin-Louis Cauchy in 1847. Herbert Robbins and Sutton Monro's 1951 work showed how to approach a solution gradually from random samples with decreasing steps. Modern training methods build on this. Early learning rules for neural networks in the 1950s and 1960s already used step size. It became a central setting after the popularization of backpropagation in 1986. The idea of automatically adjusting step size per weight was introduced by earlier methods (for example, AdaGrad in 2011). It became widespread with the Adam method presented in 2014.

Where things stand

As of late 2026, it is the default setting for training practically all neural networks. The rate value is now mostly adjusted by schedulers and adaptive optimizers during training. However, the base or peak value still typically needs to be chosen by a human or an automatic search.

Status: 2026 November

Also known as

learning rate · learning speed · learning factor · learning coefficient · step size · learning rate · step size · LR

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.