Sophia: Newton Steps with a Clip

Here is a loss with two very different directions. Across the valley, along y, the walls are steep. Along x, the floor is a gentle well, and far from the centre it flattens into a plateau that even curves slightly downward.

Gradient descent, in red, uses one step size for every direction. A step large enough to move on the plateau makes it bounce from wall to wall across the valley, and it barely crawls along the floor.

The rest of the narration is in the book's Companion.