batch size

advancedwidespread

Batch size indicates how many training examples the artificial intelligence model reviews at once before adjusting its settings once.

In plain terms

Imagine a teacher grading essays while observing what needs to be taught differently. If they change their method after every single essay, they will rush, as a single student's mistake could derail them. If they only switch after reviewing all essays, they can improve very rarely. The sensible solution is to pause, for example, every 32 essays, and then draw conclusions. This 32 is the batch size.

AI models learn in the same way. The training data is broken into smaller batches, and the model fine-tunes its internal settings slightly after each batch. The batch size therefore does not describe the model's knowledge, but the rhythm of learning.

It is important because it determines how much memory is needed for training, how fast it proceeds, and partly how well the finished model performs on new, previously unseen data. This is a so-called hyperparameter: it is not learned by the model but set in advance by the developer.

How it works

Training data is shuffled and then split into batches. A batch contains as many examples as the batch size, for instance 32, 256, or several thousand; the last batch may be smaller if the number of examples is not evenly divisible.
The model receives all examples in the batch at once, gives an answer for each, and the system calculates the average error.
This average error reveals how to adjust the model's internal numbers, or weights. Then a small correction is made, and the next batch arrives.
When all batches have been processed, a full cycle, or epoch, is completed. Training usually consists of many such cycles.
Small batches yield noisier, more erratic learning but require less memory. Large batches provide smoother training and better hardware utilization but demand more memory and require tuning of the learning rate.

An everyday example

A developer wants to fine-tune a language model on their own machine, but training stops with an "out of memory" error. They reduce the batch size from 16 to 4, and training starts, though it proceeds more slowly. From this, you see only that developing an AI service takes longer or costs more.

Who is affected

developerThis is one of the first settings to choose during training and fine-tuning, as it determines whether the task fits into the graphics card's memory. → Code

entrepreneur, executiveBatch size directly affects training time and machine rental costs, so it is worth understanding why your team requests larger hardware.

parent, student, teacherAnyone learning or teaching machine learning encounters it in the first exercises, and it is a good example of how a machine learns in batches. → Education & Family

A common misconception

MythThe larger the batch size, the better the model will be.

In realityA larger batch mainly means faster and more uniform calculation, not better knowledge. A model trained with a batch size that is too large may even perform worse on new data if other settings are not adjusted accordingly.

Limits and risks

There is no universally valid ideal value: the good batch size depends on the model, the data, and the hardware, so it must be experimented with.
Batch size is capped by memory; an excessively large batch simply will not fit on the graphics card.
The batch size cannot be adjusted in isolation: if you change it, the learning rate usually needs to be retuned as well.
Very small batches produce noisy, fluctuating learning and make poor use of expensive hardware.

Similar, but not the same

epoch – An epoch means the model has gone through the entire training dataset once, while batch size refers to the size of the chunks it processed within that.
learning rate – The learning rate determines how much the model adjusts in one step, while the batch size determines how many examples the adjustment is based on.
batch processing – Batch processing is the grouped handling of non-urgent requests submitted at once to a ready model, not a setting for training.

Related concepts

Broader

Hyperparameter, model training, machine learning

Narrower

mini-batch

On the edge

epoch, learning rate, gradient method, gradient accumulation, fine-tuning, GPU

A brief history

Per-batch learning grew out of the stochastic approximation method, whose foundations were laid by Herbert Robbins and Sutton Monro in 1951. It became a common setting with the spread of deep learning. A 2016 study (Keskar et al.) showed that models trained with very large batches may perform worse on new, previously unseen data.

Where things stand

As of late 2026, batch size is a fundamental setting for training neural networks, and it can be specified in every widely used machine learning framework. For large language models, it is often specified not in examples but in the number of tokens. If the desired batch does not fit in memory, it is often compensated for with gradient accumulation: results from several small batches are collected, and the model is adjusted only afterward.

Status: 2026 November

Also known as

batch size · batch size · batch size · Batch size · batch size · mini-batch size · minibatch size · batch size

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.