Bellman Operator: A Contraction With One Fixed Point

A small decision problem has three states, so a value function is just a vector with three entries: a point in three dimensional space.

The Bellman optimality operator takes a value vector and returns a new one. For each state, it takes the best action: its reward, plus gamma times the expected value of the next state.

The rest of the narration is in the book's Companion.