Random Fourier Features
An interactive live film: random cosine features estimate the Gaussian kernel without bias; raise their number m and watch the orange surface settle onto the blue kernel.
Narration
The blue surface is the Gaussian kernel, k of x and y, with x fixed at the origin: one at the centre, falling to zero with distance.
Bochner's theorem writes it as an average of waves: the expected cosine of omega transpose x minus y, with the frequencies omega drawn from a Gaussian. Each orange dot on the floor is one frequency.
The rest of the narration is in the book's Companion.
More from Chapter 13
Interactive1:35Normalizing Flows: Chaining Invertible MapsChapter 13
Preview1:38Independence as an Outer ProductChapter 13
Preview1:40Conditional Gaussian: A Slice of the BellChapter 13
Preview1:46Change of Variables: Same Probability, Different AreaChapter 13
Preview1:35The von Mises-Fisher Distribution: A Gaussian for DirectionsChapter 13
Preview1:42The Dirichlet Distribution: A Density on the SimplexChapter 13
Preview1:14Gaussian Mixture Model: Multimodal TerrainChapter 13
Preview1:41Whitening: PCA and ZCAChapter 13
Preview1:36Beta and Bernoulli: Updating with Pseudo-CountsChapter 13
Preview1:43ROC Curve and AUC: the Mass Above the DiagonalChapter 13
Preview1:40Hoeffding's Inequality: How Fast a Sample Mean SettlesChapter 13
Preview1:25Temperature Scaling: Calibrating a SoftmaxChapter 13
Preview1:34Mahalanobis Distance: The Ellipsoids of a GaussianChapter 13
Preview2:09Random Directions Are Nearly OrthogonalChapter 13