The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

Forward or Reverse: Filling the Jacobian

Chapter 11 · Mathematics for AI and Machine Learning · Advanced · 1:54

An interactive live film: forward mode pushes a direction onto the surface (a column of J), reverse mode pulls an output back to the floor (a row); drag v yourself, then count the sweeps.

Narration

Here is a function from the plane into space: every point of the floor is lifted onto the surface above it. Its Jacobian, on the right, has three rows, one per output, and two columns, one per input.

Forward mode pushes an input direction through the function. Push the first axis direction, and it comes out as the teal arrow along the surface: that arrow is the first column of J. One J V P fills one column.

The rest of the narration is in the book's Companion.

More from Chapter 11

Interactive1:54Backpropagation, One VJP per LayerChapter 11Preview1:27Partial Derivatives Are Two SlicesChapter 11Preview1:34The Gradient Is the Steepest DirectionChapter 11Preview1:42Jacobian Matrix: Linearizing a MapChapter 11Preview1:43The Derivative of the Matrix InverseChapter 11Preview1:32Taylor Expansion, One Term at a TimeChapter 11Preview1:30Second-Order Taylor: Where the Quadratic Model Can Be TrustedChapter 11Preview1:38A Hessian-Vector Product from Two GradientsChapter 11Preview1:34The Gauss–Newton MatrixChapter 11Preview1:44Hessian Matrix: Bowls, Saddles and Curvature DirectionsChapter 11Preview1:37Why the Hessian Is SymmetricChapter 11Preview1:26The Newton Step: Jumping to the Taylor Bowl's BottomChapter 11Preview2:36Corners, Maxima and Steps in the Chain RuleChapter 11Preview1:59GPTQ: Rounding with the HessianChapter 11Preview2:12AWQ: Activation-Aware Weight QuantizationChapter 11Preview1:41The Implicit Function TheoremChapter 11