The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

Forward Diffusion and the Heat Equation: All Distributions Become Gaussian

Chapter 21 · Mathematics for AI and Machine Learning · Beginner · 1:28

An interactive live film: an arbitrary multimodal initial distribution smooths monotonically into a standard normal bell under the heat equation, with entropy strictly increasing and higher moments decaying.

Narration

The forward process of a diffusion model adds pure noise. The probability density satisfies the heat equation, with no drift.

Solving the heat equation amounts to continuous Gaussian convolution. Each point mass spreads into a Gaussian bell, blurring fine features first.

The rest of the narration is in the book's Companion.

More from Chapter 21

Interactive1:35Fokker-Planck: A Random Cloud, a Deterministic DensityChapter 21Preview1:29Detailed Balance: Stationary Is Not the Same as StillChapter 21Preview1:27Guidance Does Not Sample the Sharpened DistributionChapter 21Preview1:50Metastability and the Kramers Escape RateChapter 21Preview1:26Reverse Time: Why We Must Learn the ScoreChapter 21Preview1:34Unification: SDEs, Probability Flow ODEs, and Fokker-Planck DynamicsChapter 21Preview1:52Discretization Changes the Stationary DistributionChapter 21Preview1:29Stein Variational Gradient Descent: Pull and PushChapter 21