The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

Mirror Descent Stays Inside the Simplex

Chapter 12 · Mathematics for AI and Machine Learning · Advanced · 1:52

An interactive live film: on the probability simplex in 3D, projected gradient descent jumps onto edges and corners while the exponentiated-gradient (mirror descent) step keeps every coordinate positive; drag the start and change the step size.

Narration

Here is the probability simplex in three dimensions: the triangle with corners e zero, e one and e two.

The loss is linear, f of x equals two x zero plus x one. The grey lines on the triangle are its level lines, and its minimum is the top corner, e two.

The rest of the narration is in the book's Companion.

More from Chapter 12

Interactive1:08Gradient Descent in an Ill-Conditioned ValleyChapter 12Preview1:28The Descent Lemma: A Bowl That Bounds Every StepChapter 12Preview1:33Heavy-Ball Momentum: Zig-Zags Cancel, the Valley Floor Adds UpChapter 12Preview1:43A Large Step Selects a Flatter MinimumChapter 12Preview2:12Sophia: Newton Steps with a ClipChapter 12Preview2:14Adam and AdamW: A Step Size for Every CoordinateChapter 12Preview1:55Backtracking Line Search and the Armijo ConditionChapter 12Preview1:50Proximal Gradient and Soft-Thresholding: Exact ZerosChapter 12Preview1:53KKT Conditions: the Saddle Point of the LagrangianChapter 12Preview1:55The Polyak-Łojasiewicz ConditionChapter 12