The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

GPTQ: Rounding with the Hessian

Chapter 11 · Mathematics for AI and Machine Learning · Advanced · 1:59

An interactive live film: two weights, the layer error as a tilted bowl over the quantization grid; rounding to nearest against GPTQ's round-compensate-round, and the reader drags the weights and turns the bowl.

Narration

To quantize a layer, GPTQ keeps its output close to the original: the error is the squared norm of X times the change in the weights. Here are two weights, and that error drawn as a bowl over them, with its bottom at the full precision weights w.

The dots on the floor are the only values the weights may take after quantization: a grid.

The rest of the narration is in the book's Companion.

More from Chapter 11

Interactive1:54Backpropagation, One VJP per LayerChapter 11Preview1:27Partial Derivatives Are Two SlicesChapter 11Preview1:34The Gradient Is the Steepest DirectionChapter 11Preview1:42Jacobian Matrix: Linearizing a MapChapter 11Preview1:43The Derivative of the Matrix InverseChapter 11Preview1:54Forward or Reverse: Filling the JacobianChapter 11Preview1:32Taylor Expansion, One Term at a TimeChapter 11Preview1:30Second-Order Taylor: Where the Quadratic Model Can Be TrustedChapter 11Preview1:38A Hessian-Vector Product from Two GradientsChapter 11Preview1:34The Gauss–Newton MatrixChapter 11Preview1:44Hessian Matrix: Bowls, Saddles and Curvature DirectionsChapter 11Preview1:37Why the Hessian Is SymmetricChapter 11Preview1:26The Newton Step: Jumping to the Taylor Bowl's BottomChapter 11Preview2:36Corners, Maxima and Steps in the Chain RuleChapter 11Preview2:12AWQ: Activation-Aware Weight QuantizationChapter 11Preview1:41The Implicit Function TheoremChapter 11