The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

Gradient Flow: Gradient Descent With Infinitely Small Steps

Chapter 20 · Mathematics for AI and Machine Learning · Beginner · 1:27

An interactive live film: on the book's narrow valley in 3D, gradient descent steps crowd onto the exact gradient-flow curve as the step shrinks, zig-zag as it grows, and fly off past two over the largest curvature.

Narration

Here is a long, narrow valley: gentle along x one, and steep along x two.

The gradient flow is the curve that always heads straight downhill: d x over d t equals minus the gradient of f. Along it, f can only decrease.

The rest of the narration is in the book's Companion.

More from Chapter 20

Interactive1:45Flow Matching: Velocity Fields and the Continuity EquationChapter 20Preview1:28From a Residual Network to a Neural ODEChapter 20Preview1:26Memory by Orthogonal Projection (HiPPO)Chapter 20Preview1:36Momentum Methods: Second-Order ODEChapter 20Preview1:34Probability Flow ODE: Deterministic Paths, Identical MarginalsChapter 20Preview1:40Backward Error Analysis: What Gradient Descent FollowsChapter 20Preview1:46Rectified Flow: Straightening by RetrainingChapter 20