The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

Superposition and Sparse Coding

Chapter 9 · Mathematics for AI and Machine Learning · Advanced · 1:25

An interactive live film: six feature directions in three dimensions; the minimum-norm solution spreads a sparse activation over all of them, sparse coding recovers the active ones.

Narration

Six feature directions, the arrows, live in only three dimensions. They cannot all be orthogonal, but they are spread so that any two have a cosine below one half.

An activation is a sparse combination of them. Here only feature two is on, with strength one and a half, and the red arrow is the activation a, equal to D times s.

The rest of the narration is in the book's Companion.

More from Chapter 9

Interactive1:34LoRA: a Low-Rank UpdateChapter 9Preview1:48LDL Is Completing the SquareChapter 9Preview1:47Rank-Revealing QR: Pick the Columns in a Good OrderChapter 9Preview1:56The QR Algorithm: Sweeping a Matrix into Schur FormChapter 9Preview1:36Non-Negative Matrix Factorization: a Cone of PartsChapter 9Preview1:33CUR: Rebuilding a Matrix from Its Own Rows and ColumnsChapter 9