long context
Long context means that the language model can consider a very large amount of text at once—sometimes equivalent to several books—in its response.
In plain terms
Every language model has a 'desk': it holds everything the model can see at once in a conversation. This includes your question, attached documents, earlier messages, and the response being drafted. This is called the context window. We speak of long context when this desk is very large: not just a few pages fit on it, but hundreds, and on the largest ones even more than a thousand.
It is like the difference between a small writing desk and a conference table. On the small desk, you can spread out only one or two sheets at a time, putting the rest back in the drawer. On the large table, the entire file is before you at once, making it easier to notice if page 3 and page 240 contradict each other.
This is why long context is useful if you want to review an entire contract, a long report, a book, or a full codebase at once. It is also important for agents, because during a long, multi-step task, numerous partial results accumulate that they must keep in mind.
A big table, however, does not mean perfect attention. The more there is on it, the easier it is for the model to overlook a detail, and the more expensive and slower the work becomes.
How it works
- The model breaks the text into small pieces called tokens; a token is roughly a syllable or a short word. Context size is measured in tokens.
- Everything the model sees counts toward the same context window: the system instructions, uploaded files, conversation history, and the response itself.
- When responding, the model relates the tokens within the context window to each other, enabling it to connect distant parts. This is the attention mechanism. The required computation grows faster than the text itself as the text length increases.
- The model itself cannot exceed its context window. If the conversation outgrows it, the chat application or agent discards the oldest parts or summarizes and compresses them. In such cases, the model truly 'forgets' the beginning of the conversation or sees only a summary of it.
- Developers therefore often select: they do not load everything, but only give the model the parts needed for the task, because this is cheaper and often more accurate.
An everyday example
You upload a 150-page condominium document package to a chatbot and ask what it says about the payment of the renovation fund. In a long-context model, the entire material is present before the model at once, so it can also signal if the founding deed and the latest general meeting resolution say something different. In a short context, the model would only see selected details, and the contradiction could easily escape its attention. It is also possible that you would receive the answer that the file is too long.
Who is affected
A common misconception
Limits and risks
- The more text you input, the slower the response. With paid, usage-based (API) access, it is also more expensive because billing is based on processed tokens, and some providers charge higher fees for very long inputs. In subscription-based chats, long content uses up the quota faster.
- With very long materials, accuracy may degrade: the model might mix up parts or overlook an important sentence, so verify key claims in the original text.
- Long context is not permanent memory. In a new conversation, the context of the previous conversation is not available to the model. If the service still 'remembers' something, that is done by a separate memory or project function, not the context window.
- What you upload goes into the provider's system, so check the data processing terms first for personal or confidential documents.
Similar, but not the same
- context window – The context window is the frame that exists for every model; long context means this frame is exceptionally large.
- memory (in an AI assistant) – Memory retains information between conversations, while context only lives within the given conversation.
- retrieval-augmented generation (RAG) – RAG first retrieves relevant details and feeds only those to the model, while in long-context scenarios, the entire material is presented to the model at once.
Related concepts
token, figyelmi mechanizmus, retrieval-augmented generation, context engineering, AI-agent
A brief history
The transformer architecture underlying today's language models was described in 2017. Early models saw only a few pages of text at once; GPT-3, for example, handled roughly two thousand tokens. In 2023, models with a context of one hundred thousand tokens appeared, followed in 2024 by widely available models with a context of one million tokens.
Where things stand
By the end of 2026, context windows of several hundred thousand tokens are standard for leading models, and several providers offer windows of one million tokens or more. Attention has shifted from sheer size to how reliably the model uses long material and what is worth loading in the first place.
In the news
Also known as
long context · long context · long context window · nagy kontextusablak · long context window · long-context model · one-million-token context · 1M context
Sources
- Vaswani et al.: Attention Is All You Need (2017)
- Liu et al.: Lost in the Middle: How Language Models Use Long Contexts (2023)
- Wikipedia: Large language model
This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.