catastrophic forgetting

advancedwidespread

The phenomenon where an AI model learns something new and suddenly loses a large portion of its previously acquired knowledge.

In plain terms

Imagine you learn Spanish, and by the time you finish, you have almost completely forgotten English. Humans also forget, but slowly and gradually, not overnight. With neural networks, however, this is a real risk: if you continue training a finished model only on new material, its old capabilities can deteriorate dramatically.

The reason is that the model's knowledge is not kept in separate drawers but is mixed within the same numbers (weights). It is like a board where you always write in the same spot: the new text overwrites the old.

Partly for this reason, models cannot simply be trained with 'daily updates' like a colleague. Developers use special methods to protect old knowledge, or store new information outside the model, for example in databases and notes.

How it works

While learning the first task, the model adjusts its internal weights so that it solves that task well.
When receiving a new task or new data, training adjusts the same weights, as it only tries to reduce the error of the new task.
Since it no longer sees the old examples, nothing signals to it that it is meanwhile ruining what it previously knew.
On older tasks, performance does not degrade slowly but drops rapidly and sharply—hence the term 'catastrophic'.
Defense can be achieved in several ways: old examples are re-mixed into training, important weights are 'slowed down', or the base model is left untouched and only a small, removable add-on module (adapter, such as LoRA) is trained on it. The adapter is not a miracle cure: when enabled, the model may perform worse on old tasks, but the original model remains intact and can be returned to at any time.

An everyday example

A company further trains its chat model on its own legal documents so that it understands contracts better. The result: it does handle contracts better, but it has become noticeably weaker than before at writing an everyday letter or answering general questions: it phrases things more stiffly, in legal language, and makes more mistakes.

Who is affected

developerWhen fine-tuning a model, you must re-evaluate its old capabilities, otherwise something may degrade unnoticed. → Code

entrepreneur, executiveA model trained on your own data may seem like a cheap idea, but poorly executed fine-tuning can reduce its general usability.

employeeThis partly explains why the model is not retrained on every conversation. What the workplace AI assistant remembers about you is stored in external memory (notes), not within the model itself. → Work & HR

parent, student, teacherA good example of how machine learning does not work like the human mind, and the model does not grow on its own with every conversation. → Education & Family

A common misconception

MythChat AI forgets the beginning of the conversation because catastrophic forgetting occurs in it.

In realityThis refers to the context window limit: the model can only see a certain amount of text at once. Catastrophic forgetting occurs during training when the model's weights change, not during conversation.

Limits and risks

The phenomenon can be reduced, but it still cannot be completely eliminated today; every defense is a compromise between retaining old knowledge and learning new knowledge.
Degradation is often hidden because developers only measure on new tasks, not on old ones.
Replaying old data is effective but expensive, and original training data cannot always be preserved due to privacy or copyright reasons.
Not every decline is catastrophic forgetting: if a provider's new model version performs worse on certain tasks than the old one, other reasons may apply, such as it being a completely retrained model.

Similar, but not the same

context window – The context window limits how much text the model can see at once in a conversation; catastrophic forgetting is the permanent loss of trained knowledge.
overfitting – In overfitting, the model memorizes the training data and generalizes poorly; here, a previously well-functioning capability is lost due to new training.
model drift – In drift, the world changes around the model while the model remains the same; here, the model itself changes, causing degradation.

Related concepts

Broader

machine learning, Neural network, model training

Narrower

–

On the edge

fine-tuning, continual learning, transfer learning, LoRA, overfitting, replay, elastic weight consolidation

A brief history

The phenomenon was described by Michael McCloskey and Neal J. Cohen in 1989, then under the name "catastrophic interference." In 2017, DeepMind researchers demonstrated with the elastic weight consolidation (EWC) method that forgetting can be significantly mitigated, giving new momentum to research.

Where things stand

As of late 2026, this is a well-known problem in every round of model training, worked around in practice mainly by training supplementary modules (adapters) and using external memory. No widely used model yet learns continuously, as humans do; this remains an open research question.

Status: 2026 November

Also known as

catastrophic forgetting · catastrophic interference · catastrophic forgetting · catastrophic forgetting · catastrophic interference

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.