The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

The Gradient Is the Steepest Direction

Chapter 11 · Mathematics for AI and Machine Learning · Beginner · 1:34

An interactive live film: turn a direction around the point and watch the slope trace a cosine; find the steepest direction yourself and see why the gradient is perpendicular to the contour.

Narration

Here is the surface, f equals x zero squared plus x one squared minus x zero times x one, with its contour lines on the floor and a point x on it.

Pick a direction d, the black arrow on the floor. The wall along d cuts the surface, and the slope of that cut at our point is the directional derivative: the gradient dotted with d.

The rest of the narration is in the book's Companion.

More from Chapter 11

Interactive1:54Backpropagation, One VJP per LayerChapter 11Preview1:27Partial Derivatives Are Two SlicesChapter 11Preview1:42Jacobian Matrix: Linearizing a MapChapter 11Preview1:43The Derivative of the Matrix InverseChapter 11Preview1:54Forward or Reverse: Filling the JacobianChapter 11Preview1:32Taylor Expansion, One Term at a TimeChapter 11Preview1:30Second-Order Taylor: Where the Quadratic Model Can Be TrustedChapter 11Preview1:38A Hessian-Vector Product from Two GradientsChapter 11Preview1:34The Gauss–Newton MatrixChapter 11Preview1:44Hessian Matrix: Bowls, Saddles and Curvature DirectionsChapter 11Preview1:37Why the Hessian Is SymmetricChapter 11Preview1:26The Newton Step: Jumping to the Taylor Bowl's BottomChapter 11Preview2:36Corners, Maxima and Steps in the Chain RuleChapter 11Preview1:59GPTQ: Rounding with the HessianChapter 11Preview2:12AWQ: Activation-Aware Weight QuantizationChapter 11Preview1:41The Implicit Function TheoremChapter 11