PEFT

in English: parameter-efficient fine-tuning

professionalwidespread

A family of methods that adapts a large AI model to a task by retraining only a fraction of its internal settings.

In plain terms

A large language model consists of billions of tiny internal settings, called parameters. If you want to teach it something new in the traditional way, such as your company's jargon, you must modify all of them. This is expensive and slow, and at the end, you must store another full-sized model.

PEFT is like not rewriting and reprinting a thick technical book, but instead sticking note pages onto the relevant pages. The book remains untouched; the notes carry the new knowledge. If the book is needed for a different topic, you only swap the note pages.

In practice, typically less than 1 percent of a model's parameters are trained, while the rest are "frozen." The result on many tasks approaches that of full fine-tuning, while requiring far less memory and enabling faster training. The resulting add-on typically ranges from a few megabytes to a few hundred megabytes, whereas a full model is several gigabytes, often tens of gigabytes.

This is why a smaller company, research group, or even a hobbyist can now customize an open model for their own purposes, even on a single powerful graphics card.

How it works

You start with a pre-trained, general-purpose base model and freeze all its original parameters, meaning they do not change during training.
You add a small, trainable extension to the model. This can be a small pair of matrices inserted alongside the layers (LoRA), a tiny module inserted between the layers (adapter), or trainable "helper" vectors placed before the input (prompt and prefix tuning).
You train the model on your own examples, but learning only modifies the extension. Therefore, less memory and time are needed, and less data may suffice.
At the end, you save the extension as a separate small file. When using it, you load it onto the base model. A LoRA-type extension can also be permanently merged into the model, so response time will not slow down.
Many such add-ons can be created for a base model, for example a legal, a customer service, and a medical one, and these can be swapped as needed.

An everyday example

From a site that shares image-generation models, you download a "LoRA" file of a few tens of megabytes, and the program starts producing images in a particular drawing style. This is an add-on made with PEFT: the base model stays the same, it just gets a small "notepad". In a similar way, an open language model can be taught a company's style and terminology.

Who is affected

developerThis allows you to customize an open model for your own tasks on affordable hardware in a short time and maintain multiple variants simultaneously. → Code

entrepreneur, executiveTraining and storing custom AI is thus much cheaper, making a model trained on proprietary data realistic even for smaller companies.

parent, student, teacherIt makes experimenting with large models feasible for university research and theses without requiring a supercomputer. → Education & Family

A common misconception

MythPEFT is merely a weaker, "poor man's" version of full fine-tuning that always yields worse results.

In realityOn many tasks, it achieves quality very close to full fine-tuning and is sometimes more stable with fewer training examples. A performance gap is expected mainly if the model must learn something very distant from its original knowledge.

Limits and risks

If the new task is very far from what the base model knows (for example, a completely new language or field), a small addition may be insufficient.
The extension works only with the base model it was built for, so switching models requires retraining.
It adopts the errors of bad or biased training data just as well and does not eliminate the base model's mistakes or biases.
An add-on downloaded from the internet with unknown origin may bring unwanted or hidden behavior into the model, so use it only from trusted sources.

Similar, but not the same

Fine-tuning – Full fine-tuning modifies all parameters of the model, while PEFT modifies only a small part or adds an extension.
LoRA – LoRA is a single specific method, currently the most popular PEFT method, while PEFT is the name of the entire method family.
RAG – RAG does not train the model; it only provides retrieved documents at response time, while PEFT modifies the model's behavior through training.

Related concepts

Broader

Fine-tuning, Transfer learning, Machine learning

Narrower

LoRA, QLoRA, Adapter, prompt tuning, prefix tuning

On the edge

Foundation model, Large language model, Quantization, Knowledge distillation, RAG

A brief history

The direction was set by adapter modules published in 2019, which showed that a language model can be adapted to a task with only a few new parameters. The best-known method today, LoRA, was introduced by Microsoft researchers in 2021. The term itself had appeared in earlier papers, but the acronym PEFT was popularized by Hugging Face's open-source library of the same name, released in early 2023.

Where things stand

As of late 2026, the default way to customize open models, for both text and image-generation models. Sharing LoRA-type add-ons is an everyday practice.

Status: 2026 November

Also known as

PEFT · parameter-efficient fine-tuning · parameter-efficient fine-tuning · parameter-efficient fine-tuning · parameter-efficient fine-tuning · parameter-efficient tuning

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.