emotion recognition

basicspreading

A technology that tries to infer from your face, voice, text or physical signals what emotional state you are in.

In plain terms

Emotion recognition promises that the machine 'reads' what you feel: whether you are happy, angry, bored, or stressed. For this, it observes your facial expressions, tone of voice, written words, and sometimes your pulse or posture.

It is like a waiter trying to guess from the guest's face whether the meal was enjoyed. They often guess correctly, but the smile may be mere politeness, and the furrowed brow may indicate anger, intense concentration, or a headache. Similarly, the machine only sees external signals, not the inner feeling.

Used in customer service, ad testing, medical research, and in cars to detect driver stress. A system monitoring driver fatigue is a related technology, but fatigue is a physical state, not an emotion. It falls under the Society and Ethics topic because if decisions about you are made based on uncertain speculation, such as in admissions or exams, it can lead to serious injustice.

How it works

The system collects signals about you: camera images of your face, audio recordings, typed text, or data from smartwatches and other sensors.
From these, it highlights measurable characteristics, such as the movement of the corners of the mouth and eyebrows, pitch, speech tempo, or word usage.
A machine learning model searches for patterns in these characteristics. The model was previously trained on thousands of examples labeled by humans ("this is joy", "this is anger").
The final result is a label or probability, such as "70 percent chance of frustration". This is an estimate, not a measurement.
A different system responds to the estimate: for example, a customer service AI agent may change its tone or hand over to a human staff member.

An everyday example

You call your service provider and explain the billing error with growing irritation. The software running in the background marks you as an "unhappy customer" based on your voice, so it prioritizes your call, and a warning appears on the agent's screen. A related technique also monitors whether you have become drowsy while driving, although fatigue is a physical state, not an emotion.

Who is affected

employeeIn the EU, monitoring employees with machine emotion recognition is generally prohibited at work, so it is worth knowing what your employer cannot do. → Work & HR

jobseeker, new graduateVideo job interviews using facial and voice analysis are among the most controversial uses, and inferring emotions in this way is banned in the EU. → Work & HR

parent, student, teacherThe use of systems analyzing students' emotions and moods in schools and exams is banned in the EU, except for medical or security purposes. → Education & Family

marketerIt can be used in advertising testing and customer experience measurement, but you must inform the affected parties, and strict data protection rules apply. → Marketing

A common misconception

MythThe machine can see on your face exactly what you feel.

In realityThe system only classifies external signals, and the same facial expression can mean different things depending on the person, situation, and culture. According to a comprehensive 2019 scientific review, emotions cannot be reliably inferred from facial movements.

Limits and risks

Its scientific basis is disputed: there is no evidence that every person would express the same emotion in the same way.
It may be less accurate for those who appeared less frequently in training data, such as people from other cultures, older individuals, or those with disabilities.
Emotional data is highly personal: if collected and stored, it can be used for surveillance and influence.
If an emotion recognition system is monitoring you, under the EU AI Act, since August 2, 2026, they must inform you, so feel free to ask.

Similar, but not the same

facial recognition – Facial recognition determines who you are, while emotion recognition attempts to estimate what you are feeling.
sentiment analysis – Sentiment analysis typically measures the positive or negative tone of texts (reviews, comments), not a specific person's momentary emotional state based on physical signs.
lie detection – Lie detection attempts to infer honesty from physical signs, while emotion recognition infers emotional state, but the reliability of both is heavily debated.

Related concepts

Broader

affective computing, biometrikus adatok, Computer vision, machine learning

Narrower

facial expression analysis, Speech-based emotion recognition

On the edge

facial recognition, sentiment analysis, algorithmic bias, EU MI-rendelet, workplace monitoring, driver monitoring system

A brief history

The theoretical background of the field comes from Paul Ekman's research on basic emotions, which became known in the 1970s. The research direction for machine implementation was launched by MIT researcher Rosalind Picard with her 1995 study "Affective Computing". The EU's AI Regulation adopted in 2024 already specifically regulates and partially bans it.

Where things stand

Increasingly common in customer service, cars, and market research. However, in the EU, since February 2, 2025, it is generally prohibited in workplaces and educational institutions, with exceptions for medical and security purposes.

Status: 2026 November

In the news

Appears in 2 Agent Age stories; the curve shows weekly occurrence in the news magazine.

EU AI Act: which school and university AI counts as high-riskOct 14, 2026 · Education & Family
Hungary's AI Authority issues guidance mentioning emotion recognition in educational institutionsAug 6, 2026 · Education & Family

Also known as

emotion recognition · emotion recognition system · emotion detection · emotional AI · emotion recognition · emotion AI · facial emotion recognition

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.