diffusion model

advancedwidespread

An AI model that starts from random noise and creates new images, videos, or audio by cleaning it step by step.

In plain terms

Imagine sprinkling sand onto a photo repeatedly until nothing is visible. A diffusion model learns the reverse: how to sweep away the sand layer by layer so that a meaningful image remains. Once it masters this, it can generate images from pure 'sand', including ones that never existed before.

This technique underlies many current image generators and most video generators, and is also used in music and sound generation. When you type 'watercolor painting of a sailboat on Lake Balaton', your text guides the denoising: at each step, the model adjusts the noise in a direction that makes the result increasingly match your description.

It works differently from the language models behind text chatbots. Those write the answer word by word, while the diffusion model refines the entire image at once, from a blurry patch to the details.

How it works

Training, first half: the system takes a vast number of real images and gradually adds increasing amounts of random noise to each until the image completely disappears.
Training, second half: a neural network practices estimating how much and what kind of noise was added to a noisy image, i.e., how to make it cleaner in one step.
Generation: the model starts from pure random noise and gradually removes a small amount in a few dozen steps, so shapes and then details emerge from the blotches.
Control: your text instruction is converted into numbers by another model, which steers the cleaning process in the requested direction at every step.
Acceleration: modern systems usually do not work directly on pixels but on a compressed version of the image, reconstructing it into full resolution only at the end.

An everyday example

You create a birthday invitation and type into an image generator: 'cartoon-style fox holding a cake'. Within seconds, the image is ready. If the program displays intermediate states, showing the fox gradually emerging from a grainy, blurry patch, you are observing the denoising steps of a diffusion model. Not all applications show this, and not all image generators use diffusion.

Who is affected

marketerYou can generate campaign images, product photo variations, and short videos in minutes, but legal and quality checks remain your responsibility. → Marketing

parent, student, teacherChildren encounter generated images and videos daily, so it is important to teach them that a realistic-looking image is not proof. → Education & Family

lawyer, citizenMuch of today's misleading fake imagery depicting real people, and many copyright disputes, are linked to this technique. → Law

developerBuilding on open models, you can add your own image, audio, or video generation function to your application, even as an agent tool. → Code

A common misconception

MythImage generators cut out pieces from existing images and paste them together as a collage.

In realityThe model does not store images but patterns distilled from training images, and reconstructs every image anew from noise. It is rare for a frequently seen training image to be reproduced almost exactly, yet the copyright issue remains real.

Limits and risks

It does not understand what it draws: it may make mistakes with hands, text within images, the number of objects, or the laws of physics, although newer models have improved significantly in these areas.
It adopts the distortions of training images, so it can produce stereotypical representations, for example regarding professions or ethnic groups.
It can create misleadingly realistic fake images and videos, so verifying a picture's authenticity today requires checking the source rather than visual inspection.
Due to the many steps, it is computationally intensive; video generation is particularly expensive and energy-intensive.

Similar, but not the same

large language model – The language model mostly predicts the next text segment sequentially, while the diffusion model refines the entire image at once from noise. Experimental language models already exist that operate on the diffusion principle.
generative adversarial network (GAN) – A GAN generates an image in a single step using two competing networks, while a diffusion model does so in many small cleaning steps, which is slower but allows for more stable training and more varied results.
generative AI – Generative AI is the collective term for systems that produce content; the diffusion model is just one method within it.

Related concepts

Broader

generative AI, machine learning, deep learning, neural network

Narrower

latent diffusion model, text-to-image generation model, text-to-video generation model

On the edge

generative adversarial network, large language model, deepfake, multimodal model, prompt

A brief history

The basic idea was described in 2015 by Jascha Sohl-Dickstein and his colleagues, drawing from the physical diffusion process. The breakthrough came with the 2020 study "Denoising Diffusion Probabilistic Models" (Jonathan Ho, Ajay Jain, Pieter Abbeel). In 2022, the general public also became familiar with it through DALL·E 2 and the open Stable Diffusion.

Where things stand

As of late 2026, it is one of the dominant techniques for image, video and audio generation, built into mobile apps, design tools and office software. Some image generators, however, work on a different principle, such as step-by-step generation similar to language models. Development is moving towards faster versions that need fewer steps, and longer, more consistent videos.

Status: 2026 November

Also known as

diffusion model · diffusion models · diffusion model · Diffusion AI · Denoising diffusion model · DDPM · latent diffusion · latent diffusion

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.