The Four Fundamental Subspaces

Chapter 3 · Mathematics for AI and Machine Learning · Advanced · 1:47

An interactive live film: input and output R^3 side by side in 3D; the row space and null space meet at a right angle, A drops the null part and carries the row part onto the column space, and the left null space is never reached.

Narration

A three by three matrix A of rank two takes vectors in the input space, on the left, to the output space, on the right. The faint arrows are the usual axes of each space.

On the left, the blue plane is the row space, spanned by the rows of A. The red line is the null space. Every null vector is perpendicular to every row, so the line meets the plane at a right angle.

So every input x splits into two parts: x r in the plane, and x n on the line.

A sends the null part to zero. Watch x slide up and down the red line: the output on the right does not move at all.

The plane part travels across and lands on the green plane, the column space, which holds every possible output. A matches the two planes one to one.

The amber line on the right is the left null space, perpendicular to the column space. No output ever reaches it. A target b off the plane cannot be hit exactly: the closest output is its shadow on the plane, and the gap points along the amber line.

Your turn. Drag x r in the blue plane, and move the null part with the slider. Which of the two changes the output?

Each space splits into two perpendicular pieces, a plane and a line: two dimensions plus one, three in all.

More from Chapter 3

Preview1:45Orthonormal Matrices: Lengths Survive, Even through Many LayersChapter 3Preview1:36Euler's Formula: Complex Multiplication Is a RotationChapter 3Preview1:41Linear Regression Uses All Four SubspacesChapter 3