The Polyak-Łojasiewicz Condition
An interactive live film: a non-convex loss that satisfies the PL inequality, both sides of the inequality live at a point you drag, and gradient descent's gap falling geometrically on a log-scale chart.
Narration
Here is a loss that is not convex: f of w equals w zero squared, plus three times sine squared of w zero, plus w one squared.
Over the purple bands on the floor, the surface curves downward along w zero. Still, its only critical point is the minimum at the origin, where f star is zero.
The rest of the narration is in the book's Companion.
More from Chapter 12
Interactive1:08Gradient Descent in an Ill-Conditioned ValleyChapter 12
Preview1:28The Descent Lemma: A Bowl That Bounds Every StepChapter 12
Preview1:33Heavy-Ball Momentum: Zig-Zags Cancel, the Valley Floor Adds UpChapter 12
Preview1:43A Large Step Selects a Flatter MinimumChapter 12
Preview2:12Sophia: Newton Steps with a ClipChapter 12
Preview2:14Adam and AdamW: A Step Size for Every CoordinateChapter 12
Preview1:50Proximal Gradient and Soft-Thresholding: Exact ZerosChapter 12
Preview1:53KKT Conditions: the Saddle Point of the LagrangianChapter 12
Preview1:52Mirror Descent Stays Inside the SimplexChapter 12