The ELBO: a Hill under the Evidence

A tiny latent variable model: z is standard normal, and x given z is normal around z with variance one half. We observe x equal to one and a half. The evidence, p of x, integrates over every z.

Pick any Gaussian q for z, with a mean and a standard deviation: a point on this floor. Above it stands its evidence lower bound, the expected log joint minus the expected log q. Over the floor, the ELBO is a hill.

The rest of the narration is in the book's Companion.