Conjugate Gradient: Two Steps to the Bottom

Here is the bowl phi of x equals one half x transpose A x minus b transpose x, with A equal to four, two, two, two, and b equal to two, zero. Its lowest point, the green star, is x star equals one, minus one: the solution of A x equals b.

Start at the origin. Steepest descent moves along the residual, the downhill direction, and stops where the line just touches a contour. Then it turns by a right angle. The orange path zig zags in ever smaller steps.

The rest of the narration is in the book's Companion.