figyelemmechanizmus
in English: attention mechanism
The attention mechanism is the method by which an AI model, such as a language model, decides which parts of the input (for example, a text) are important for interpreting a specific part, such as a word.
In plain terms
When you read: "The dog did not cross the road because it was too tired," you know without thinking that the dog was tired, not the road. This is not self-evident to a machine. The attention mechanism is the process by which a model examines every other word for each word and decides how much each contributes to understanding.
It is like talking in a crowded restaurant: you hear everyone, but you tune in only to the one or two voices that matter at that moment. The model similarly "turns up the volume" on the important words and turns down the rest, and it does so anew, and differently, for every single word.
Earlier systems processed text word by word, and by the time they reached the end, the beginning had faded from their memory. With the attention mechanism, the model can directly refer back to any earlier part of the text, thus finding even distant connections. This is why a chatbot today can summarize a long letter or follow the thread of a multi-turn conversation.
How it works
- The model breaks the text into small units called tokens and converts each into a number sequence that describes its meaning and position in the text.
- Every token asks a question of the others: who helps me interpret? The model calculates pairwise how well two tokens fit together.
- From these values, weights are derived: important words get high weights, while irrelevant ones get nearly zero.
- The token's new, richer meaning is composed of a weighted blend of other tokens, so the word 'tired' incorporates that it concerns a dog.
- This happens with many parallel "attention heads" and repeated across many stacked layers. During learning, individual heads become sensitive to different patterns, such as grammatical relationships or references, but this division of labor is not sharp and not pre-set.
An everyday example
You enter 'Vár a vár előtt' into a translation program. The program knows that the first 'vár' is a verb and the second is a building because the attention mechanism infers from the other words in the sentence which meaning fits. The same happens when the chatbot knows who you mean by 'ő' even in your tenth question.
Who is affected
A common misconception
Limits and risks
- With very long texts, attention 'dilutes': the model may overlook details hidden in the middle, so it is worth highlighting important points.
- Computational demand rises steeply with text length, making processing long inputs slower and more expensive.
- Attention weights do not provide a reliable explanation for why the model responded as it did; what it "looked at" is not yet justification.
- Attention can only focus on what is present in the input: if data is missing, the model does not notice the gap but may invent something to fill it.
Similar, but not the same
- transformer – The transformer is the entire model architecture, while the attention mechanism is its most critical component, roughly like the engine in a car.
- context window – The context window specifies how much text fits into the model at once, while the attention mechanism determines how much weight is assigned to what within that.
- emberi figyelem – Humans pay attention consciously and with limited capacity, while the model simply compares words to each other through calculation. In text-writing language models, every word is only related to the ones preceding it.
Related concepts
Neural network, deep learning, Natural language processing
self-attention, multi-head attention, keresztfigyelem
transformer, large language model, context window, token, embedding
A brief history
The attention mechanism as understood today was introduced in 2014 by Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio to improve machine translation. At that time, it complemented networks that processed text word by word. In 2017, Google researchers' paper "Attention Is All You Need" showed that a model could work without such word-by-word progression, based fundamentally on attention: thus the transformer, the basis of today's language models, was born.
Where things stand
As of late 2026, the vast majority of significant language, image, and audio models are built on attention mechanisms. Some newer models combine attention with cheaper layer types, such as state-space models. Research mainly focuses on how to make attention cheaper and more reliable on long texts.
In the news
- Government decree on National Institute of Pharmacy monitors medical AI decision-support softwareOct 30, 2026 · Health
- AI visibility market grows: Coty–Brandlight, Marketbridge acquisition, Joveo and GEO start-upsOct 26, 2026 · Marketing
- Colorado draft AI rules and New York AI safety proposals draw attentionOct 26, 2026 · Jog
- Indian workers film their own work to train AIOct 25, 2026 · Work & HR
- AI backlash grows as small businesses, local communities and shoppers test brand trustOct 19, 2026 · Marketing
- ChatGPT vs Google: analysis points to a 27x conversion gap for SEO professionalsOct 19, 2026 · Marketing
- Judges see promise and peril as AI moves into the courtsOct 12, 2026 · Jog
- FDA has cleared more than 1,600 AI medical devices; warning letter flags software updatesOct 12, 2026 · Health
Also known as
figyelemmechanizmus · figyelmi mechanizmus · attention mechanism · attention · attention layer
Sources
- Vaswani et al.: Attention Is All You Need (2017, arXiv)
- Bahdanau, Cho, Bengio: Neural Machine Translation by Jointly Learning to Align and Translate (2014, arXiv)
- Wikipedia: Attention (machine learning)
This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.