Score Function: Arrows That Point Uphill on the Log Density
An interactive live film: a two-bump density as a 3D surface with its score field on the floor; the arrows vanish at the peaks and the saddle, ignore the normalizing constant, and smooth out as denoising noise sigma grows.
Narration
Here is a density with two bumps, drawn as a surface over the plane.
The score is the gradient of the log density with respect to x. At every point of the floor it is an arrow, and every arrow points uphill on the log density, toward a bump.
The score never needs the normalizing constant. It disappears when we take the log and differentiate, so a model can learn the score without computing an integral.
At the peaks, and at the saddle between them, the score is zero. Far away the arrows grow long, and point straight back toward the data.
Denoising score matching learns the score of the data blurred by Gaussian noise of size sigma. Raise sigma: the bumps melt together, and the arrows become smooth and point toward the middle.
Your turn. Drag the point across the floor, and change the noise level with the slider.
Follow these arrows, with a little noise added, and you get Langevin sampling.
