The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

The Discounted Return: Now or Later

Chapter 15 · Mathematics for AI and Machine Learning · Beginner · 1:32

An interactive live film: three early points against five late ones; every reward k steps away is worth γᵏ of itself today, the return G adds them up, and the discount factor γ decides which path wins.

Narration

Two ways to collect rewards. The teal path pays one point at each of the first three steps. The orange path pays nothing for a while, then five points at step eight.

A reward that arrives k steps from now counts gamma to the k times as much. The faint bars are the rewards, and the solid bars are what they are worth today.

The rest of the narration is in the book's Companion.

More from Chapter 15

Interactive1:41Value Iteration and Policy Iteration: Many Small Steps or a Few Big JumpsChapter 15Preview1:39The Bellman Equation as a Linear SystemChapter 15Preview1:49Bellman Operator: A Contraction With One Fixed PointChapter 15Preview1:34Continuous-Time Markov Chains: Probability Flowing on a TriangleChapter 15Preview1:58The KL-Regularized Optimal PolicyChapter 15Preview1:51The Deadly Triad: When TD Learning ExplodesChapter 15