lip-syncing

in English: lip sync

basicwidespread

An AI procedure that retrospectively adjusts the lip movements of a person or figure in a video to match an audio recording, making it appear as if they are actually saying it.

In plain terms

With dubbed films, you have surely noticed that the actor's mouth forms something different from what you hear. Traditional dubbing mitigates this by adjusting the Hungarian text to the mouth movements. AI-based lip synchronization reverses this: the text remains, and the mouth on the screen is redrawn to match the audio.

It is like a very skilled puppeteer who moves the puppet's mouth by listening to the voice, except here the puppet is a real person's face in the recording. The system repaints the lips, teeth, and jaw frame by frame, leaving the rest of the video untouched.

It is used for: translating films and educational videos into other languages, animating digital avatars, and animating cartoon characters. The same technique is also a main tool for deceptive fake videos, so it is worth knowing it exists.

How it works

The system breaks the audio recording into tiny time slices and extracts when which speech sound is audible.
Each sound has a characteristic mouth shape (different for "o", "m" and "f"), but several sounds look identical on the lips, such as "m", "b" and "p". The model learned from a huge number of videos of people speaking which mouth shape fits which sound.
It finds the face in the video and tracks its position, head rotation, and lighting.
It regenerates the area around the mouth frame by frame so that it takes a form matching the sound, while preserving the person's skin tone, teeth, and facial hair.
Finally, it blends the new mouth region into the original image. The model was trained from the start so that the motion is continuous and moves in exact sync with the audio.

An everyday example

You watch a foreign instructor's video on your phone, and they speak to you in Hungarian, their mouth forming the Hungarian words precisely, even though the person does not know a word of Hungarian. A translation tool converted the video: it translated the text, had it re-spoken in the instructor's own voice, and then adjusted the mouth movements to match.

Who is affected

everyoneA video with believable lip movements no longer proves that someone actually said what you hear from them.

marketerFrom a single recorded ad or product video, you can create multi-language versions without reshooting. → Marketing

parent, student, teacherCourse materials and lectures are easier to translate, but students must also learn that videos can be manipulated. → Education & Family

lawyer, citizenIf a video is made using someone's face and voice without their permission, it raises personality rights and data protection issues. Since August 2026, misleadingly lifelike manipulated videos (deepfakes) in the EU must be disclosed as artificially created or modified. → Law

A common misconception

MythLip-syncing is the same as deepfake, so it is inherently meant for deception.

In realityThis is a technique used largely ethically by the film industry, education, and advertising, with the participants' consent. It becomes abuse when someone puts words in another's mouth without permission or conceals the manipulation.

Limits and risks

With a turned head, a hand near the mouth, a microphone, or fast movement, the area around the mouth often blurs or breaks apart.
Teeth, the tongue, and very emotional facial expressions often remain unnatural, with the mouth and eyes not conveying the same feeling.
Scammers use the faces of well-known people to create fake investment offers and political messages, so a video alone should not convince you of anything.
Use another person's face and voice only with their permission. Under the EU AI Act, since August 2, 2026, artificially generated or manipulated video depicting a real person in a deceptively lifelike manner (deepfake) must be labeled as not original; for obviously artistic or satirical works, a lighter form applies.

Similar, but not the same

deepfake – Deepfake is the collective term for all deceptively lifelike images, audio, and video created or modified with AI; lip-syncing is just one technique.
voice cloning – Voice cloning imitates someone's voice, while lip-syncing adjusts the image to the sound; the two are often used together.
arccsere – Face swapping replaces the entire face with that of another person, while lip-syncing keeps the original face and only rewrites the mouth movement.

Related concepts

Broader

generative AI, video generation, synthetic media

Narrower

–

On the edge

deepfake, voice cloning, digital avatar, speech synthesis, arccsere, AI-szinkron, talking head generation

A brief history

Synchronizing lip movement with audio has been a fundamental task in filmmaking since the advent of sound films. Using machine learning, the Video Rewrite research system was the first, in 1997, to rewrite a person's lip movements on an existing recording to match new audio, and the Wav2Lip model, published in 2020, made the technique widely accessible for any face and voice.

Where things stand

By the end of 2026, video translation and avatar-creation services are a basic feature, and on simpler recordings the result is barely noticeable to a layperson. It still makes mistakes in complex scenes, and because of misuse, labeling is increasingly mandatory in many places.

Status: 2026 November

Also known as

lip sync · lip-sync · lipsync · lip syncing · lip-sync · ajakszinkron · AI lip sync · audio-driven lip synchronization

Sources

Prajwal et al.: A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild (2020)

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.