Tweedie's Formula: The Denoiser Is the Score
An interactive live film: noisy observations off a clean circle manifold; the optimal denoiser steps along the score scaled by sigma squared, projecting corrupted samples straight back onto the data manifold.
Narration
In denoising score matching, a network learns to remove noise from corrupted data. Tweedie's formula reveals why this simple task recovers the exact score of the noisy distribution.
Look at the clean data circle in blue. Adding Gaussian noise disperses samples off the manifold into the surrounding plane. The orange dot is our noisy observation x tilde.
The rest of the narration is in the book's Companion.
