SVD: Rotate, Stretch, Rotate

Every matrix A factors as U times sigma times V transpose. To see what that means, watch what A does to the unit sphere, with three special directions on it: v zero, v one and v two.

First, V transpose. It is orthogonal, so it only rotates. The sphere stays a sphere, and the three directions turn onto the coordinate axes.

Then sigma stretches each axis by its singular value: sigma zero, the largest, then sigma one, then sigma two. The sphere becomes an ellipsoid lined up with the axes.

Finally U rotates once more. The axes of the ellipsoid now point along u zero, u one and u two, and their half lengths are the singular values. A times v i equals sigma i times u i.

Now shrink the smallest singular value to zero. The ellipsoid flattens into an ellipse. A has rank two, and its outputs fill only a plane, the column space.

Your turn. Scrub through the three steps with the stage slider, and change the smallest singular value.

Rotate, stretch, rotate. Every matrix does just this, and its singular values say how much it stretches in each direction.