Rayleigh Quotient: The Height of a Quadratic Form on the Unit Circle

For a symmetric matrix A, the quadratic form x transpose A x is a surface over the plane. Here both eigenvalues are positive, so the surface is a bowl.

Its axes are the eigenvectors of A, perpendicular to each other, in red and blue. Along each axis the surface is a parabola, and its steepness is the eigenvalue.

Now walk around the unit circle. On the circle, the height of the surface is exactly the Rayleigh quotient, x transpose A x over x transpose x.

The height rises and falls twice per turn. It is highest above the red eigenvector, at the largest eigenvalue, and lowest above the blue one, at the smallest. Every other direction lies in between.

Now lower the smaller eigenvalue below zero. The bowl turns into a saddle. The form is indefinite, positive in some directions and negative in others, and the Rayleigh quotient changes sign around the circle.

Your turn. Turn the arrow around the circle, and find the direction with the largest Rayleigh quotient. Then change the smaller eigenvalue.

Positive definite means every direction goes up. The Rayleigh quotient always stays between the smallest and the largest eigenvalue, and it reaches them at the eigenvectors.