Hyperparameter

advancedwidespread

A hyperparameter is a setting that the model does not learn but is specified externally before training (by the developer or an automatic search program), determining how the model learns and how large it should be.

In plain terms

Imagine baking bread. The dough transforms itself in the oven, but you must decide in advance at what temperature and for how long to bake it. In machine learning, hyperparameters are the same: the oven's knobs, not the bread.

During training, an AI model sets many internal numbers for itself based on the data. In large models, this number can reach several billion. These are called parameters. Hyperparameters, however, are not learned by the model but are chosen in advance by the developer: for example, how large the steps should be in learning, how many times the data should be reviewed, or how large the model itself should be.

It is important because the same data can yield a useless model with poor settings or an excellent one with good settings. Good values are rarely calculable and usually must be experimented with.

How it works

Before training, the developer sets parameters such as the learning rate (how large the steps should be for model improvement), the number of training epochs, and the model size.
The model learns from the data with these settings while tuning its own internal parameters.
The trained model is evaluated on a held-out validation dataset. Since the model did not see this data during training, it reveals how well it generalizes.
The developer modifies the settings and retrains. This is called hyperparameter tuning, which is now often performed by automatic search programs.
The final model is prepared with the settings that yield the best results. Its performance is ultimately verified on a new test dataset that has not been used at all before.

An everyday example

When you adjust the 'creativity' or 'temperature' slider in a chatbot's settings, you encounter a related concept: a setting not learned by the model but provided from outside. The difference is that this regulates the response rather than the training. The same model produces more predictable answers at low values and more varied, sometimes erratic, answers at higher values.

Who is affected

developerDuring model training or fine-tuning, it must choose these values, and the quality of the result and the computational cost depend on this. → Code

entrepreneur, executiveIf your company trains a model on its own data, tuning the settings takes time and machine time, which must be accounted for in the budget.

parent, student, teacherFor anyone learning or teaching machine learning, this is one of the first concepts to distinguish from a parameter. → Education & Family

A common misconception

MythA hyperparameter is a particularly large or particularly important parameter.

In realityHere 'hyper' means the setting is one level above the parameters: the model does not learn it, but it is specified from outside and regulates the learning process.

Limits and risks

There is no one-size-fits-all recipe: what works on one dataset and model may yield weak results on another.
Tuning is expensive because every combination tried means another training run, which for large models takes significant compute time and energy.
If settings are refined until the best numbers appear on test data, the model may perform worse in real-world use than the metrics suggest.
Published results are hard to replicate if the authors do not share the exact settings used.

Similar, but not the same

parameter – The model learns the parameter from the data itself, while the hyperparameter is set by a human or a search program before training.
fine-tuning – Fine-tuning is further training of a finished model on new data, while hyperparameter tuning is the trial-and-error search for training settings.
prompt – A prompt is a text instruction for the model during use, while a hyperparameter is a numerical or structural setting that typically governs training.

Related concepts

Broader

machine learning, model training

Narrower

learning rate, Batch size, epoch count

On the edge

parameter, fine-tuning, overfitting, validation dataset, AutoML, temperature

A brief history

The term comes from Bayesian statistics, where it referred to numbers that describe a prior assumption about another parameter (the so-called prior distribution). In machine learning it was already used in the 1990s, and with the rise of deep learning in the 2010s it became an everyday technical term.

Where things stand

As of late 2026, it is a basic concept of machine learning, present in the training of every model. Tuning is increasingly done by automated tools. For large language models, because training is so costly, only a few combinations can be tried at full size. Settings are therefore often found on smaller models and then carried over to the large model.

Status: 2026 November

Also known as

Hyperparameter · hyperparameters · hyperparameter · hyper-parameter · hyperparameter tuning · hyperparameter optimization · hyperparameter tuning · HPO

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.