kis nyelvi modell (SLM)

in English: small language model

basicspreading

A text-understanding and text-writing AI model that is much smaller than large models, allowing it to run cheaply and quickly on phones, laptops, or corporate machines.

In plain terms

A small language model does the same as its larger siblings, such as the models behind ChatGPT: it understands and writes text. It is just much smaller: it stores less 'knowledge', requires less memory and power, so it does not need a massive data center.

Imagine it like a one-volume mini-encyclopedia next to a twenty-volume large encyclopedia. The mini-encyclopedia does not contain everything, but it is with you, you can flip through it immediately, and it is sufficient for most daily questions. The large encyclopedia knows more, but you do not take it with you on the tram.

It is good for solving well-defined tasks quickly and cheaply: summarizing emails, correcting text, filling out forms, and categorizing questions. Since it can run on your own device, your data does not necessarily leave your machine, and it can work without internet.

There is no sharp boundary for what counts as "small": models that run on an average phone or computer are typically called this. What is small today was considered large a few years ago.

How it works

The model's 'knowledge' is stored in numbers, so-called parameters. A small model typically contains several hundred million to roughly ten billion of these, while the largest contain multiples of this, even several hundred billion or more.
A small model is often trained with the help of a large model: the small one learns by imitating the large model's answers, thus preserving part of the large model's knowledge in a much smaller size. This is called knowledge distillation. Additionally, carefully selected, high-quality training text plays a significant role.
The size is further reduced by tricks such as storing numbers less precisely, rounded (quantization). This makes the model smaller and faster, and with moderate rounding, its quality degrades only slightly.
They are often fine-tuned for a narrow area, such as customer service emails or medical texts. There, they can approach the performance of large models because they do not need to understand everything.
The ready-made model runs directly on your device or the company's own server. In more complex systems, the small model handles routine tasks, and only difficult questions are forwarded to a large model.

An everyday example

Some newer phones offer, even in airplane mode, to summarize long messages or rewrite your reply to be more polite. In such cases, it is not a distant data center working, but a small language model running directly on the device.

Who is affected

everyoneAI functions built into your phone, laptop, and car increasingly rely on small language models, often keeping your data on the device.

entrepreneur, executiveWith a small model, you can automate routine tasks at a fraction of the cost on your own machine, without sending sensitive company data to an external provider.

developerYou must decide which tasks are suitable for a small model and when it is worth calling a more expensive, larger model, for example during certain steps of an agent's execution. → Code

patient, doctorHealthcare data is particularly sensitive, so it is a major advantage if the text-processing model runs on the hospital's own machines. → Health

A common misconception

MythA small language model is merely a weaker, cheaper imitation of a large model, not worth using for serious work.

In realityOn well-defined tasks, a fine-tuned small model can approach the performance of large ones, while being faster and cheaper, and if run locally, it better protects data. However, in general knowledge and complex, multi-step reasoning, it truly lags behind them.

Limits and risks

It has less general knowledge, so it makes mistakes sooner than a large model in rare topics, with long texts, and in complex reasoning.
It is just as capable of confidently stating falsehoods as large models, so verify its important answers.
On smaller languages, including Hungarian, it is often noticeably weaker than in English because it was trained on fewer texts in these languages, and due to its small size, less 'space' is allocated to them.
The fact that a model is small does not mean your data stays on your machine: this depends on where the provider runs it, which is worth investigating.

Similar, but not the same

large language model – A large language model works with more parameters and broader knowledge, but typically runs in a data center, making it more expensive and slower.
on-device AI – On-device AI specifies where the model runs, while the small language model specifies its size; the two often go together, but a small model can also run in the cloud.
knowledge distillation – Knowledge distillation is a method for creating a small model, while the small language model itself is the end result.

Related concepts

Broader

language model, generative AI, foundation model

Narrower

–

On the edge

large language model, on-device AI, knowledge distillation, quantization, fine-tuning, distilled model, quantized model

A brief history

An early milestone in the shrinking of language models was DistilBERT, introduced in 2019. The term 'small language model' itself spread in 2023–2024, when models designed for phones and laptops appeared one after another as counterparts to large language models, such as Microsoft's Phi family.

Where things stand

By the end of 2026, most major AI developers offer a small model too, and many newer phones and laptops run them built in. Companies increasingly pair them with large models, for example for the routine steps of agents.

Status: 2026 November

Also known as

kis nyelvi modell · small language model · kicsi nyelvi modell · kompakt nyelvi modell · small language model · SLM · small LLM · SLM-ek

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.