voice cloning

basicwidespread

Voice cloning is an AI procedure that can have any text spoken in a person's voice based on a short recording of their voice.

In plain terms

Imagine a voice-mimicking comedian who can make anyone speak after listening for just a few minutes. Voice cloning does the same with machines: it learns your timbre, intonation, and speech tempo from a short recording, then pronounces sentences in your voice that you never actually said.

It is useful for many things. Audiobooks can be created in the author's voice, films can be dubbed so the actor's own voice speaks in another language, and the voice of someone who lost it due to illness can be restored.

Traditional text-to-speech speaks in a pre-prepared, "nobody's" voice. A voice clone copies the voice of a specific, real person, which makes it valuable and also dangerous: it can be used for fraud and deception just as easily.

How it works

The system is pre-trained on thousands of hours of speech from many people, so it generally learns how the human voice is structured.
It receives a sample from the target person's voice. Today's tools need only a few seconds or minutes of recording for this.
It creates a 'voiceprint' from the sample: a numerical sequence describing timbre, pitch, rhythm, and characteristic pronunciation.
When you input text, the model generates speech based on the voiceprint, including stress, pauses, and even emotional coloring.
More advanced systems can do this in real time, altering a speaker's voice to sound like someone else during a live phone call.

An everyday example

You receive a call from your "grandchild" or "child" in a crying voice, claiming they were in an accident and need an immediate money transfer. The voice sounds identical to the real person, but it was generated by scammers using an audio track from a video uploaded to social media. In such cases, hang up and call the person back on their known number.

Who is affected

everyoneAnyone's voice can be cloned if there is a public video or audio recording, and anyone can be called using a cloned voice of an acquaintance.

bank customer, investorScammers use cloned voices to impersonate relatives, bank officials, or company executives to induce transfers, and voice-based authentication has also become vulnerable. → Money

marketerAd voice, dubbing, and multilingual video can be produced quickly and cheaply with it, but only with the explicit consent of the voice owner. In practice, this means a permission recorded in a contract. → Marketing

lawyer, citizenA recording is personal data, and one's own voice is part of personality rights, so unauthorized cloning is generally illegal. The evidentiary value of recordings has also become questionable. → Law

A common misconception

MythOnly professionals can create voice clones from long studio recordings.

In realityNowadays, a few seconds of voice recorded on a phone is enough, and it can be done by anyone using cheap or free online tools.

Limits and risks

A fake is increasingly hard to tell from the original by ear, so don't decide based on voice alone: if a request seems suspicious, call the person back, or agree on a secret password within the family.
Programs that detect fake voices are not reliable, especially on noisy, compressed phone recordings, so 'real according to the detector' is not proof.
As a rule, you may clone another person's voice only with their explicit consent. Without it, you may commit an infringement even if you meant it as a joke.
The quality of Hungarian-language clones has improved significantly, but stress, rare words, and strong emotions often still reveal the machine.

Similar, but not the same

speech synthesis – Speech synthesis reads a text aloud in any machine voice, while voice cloning imitates the voice of a specific, real person.
deepfake – A deepfake is a deceptive forgery that can be an image, video, or audio. Voice cloning itself is just a technique that can be used for legitimate purposes. It becomes a voice deepfake only when used to deceive someone.
speech recognition – Speech recognition converts spoken speech into text; voice cloning does the reverse, generating speech in someone's voice from text.

Related concepts

Broader

generative AI, synthetic media, speech synthesis

Narrower

real-time voice conversion, audio deepfake, AI-szinkron

On the edge

deepfake, speech recognition, watermarking, phishing, voice-based agent

A brief history

The foundation of modern voice cloning is neural speech synthesis, with a milestone being DeepMind's WaveNet model introduced in 2016. In 2019, a fraud case became known where a company executive's cloned voice was used to extract money. In 2023, Microsoft's research model VALL-E demonstrated that a person's voice could be imitated from a three-second sample.

Where things stand

By the end of 2026, voice cloning is a mass-market product: present in consumer applications, dubbing studios, and phone AI agents alike, while fraud committed with it has become commonplace. The EU Artificial Intelligence Act (Article 50) requires that machine-generated audio be marked in a machine-detectable way and that voice deepfakes be disclosed to the listener. These rules apply from 2026, but enforcement is still in its early stages.

Status: 2026 November

In the news

Appears in 6 Agent Age stories; the curve shows weekly occurrence in the news magazine.

Deepfake executive scams: $95 million loss, FBI data, INTERPOL and KeyBank warningsOct 19, 2026 · Money
Banks seek customer trust against AI voice cloning as SBI warns and call protection spreadsOct 12, 2026 · Money
Lithuanian lawmakers propose legal protection against AI voice cloningOct 12, 2026 · Jog
AI voice clone and deepfake used to trick Italian bank into wiring €95 millionOct 5, 2026 · Money
Voice-cloning scams: advice for consumers and new AI-driven fraud methodsOct 5, 2026 · Money
AI voice cloning behind new wire fraud scam targeting law firmsAug 4, 2026 · Money

Also known as

voice cloning · voice clone · voice cloning · AI voice clone · voice cloning · voice clone · audio deepfake · hang-deepfake

Sources

Wang et al. (2023): Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers (VALL-E)

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.