Speculative Decoding: Exact Sampling With a Draft Model

Here are six tokens. The blue bars at the back are the large model's next token distribution p. The orange bars in the middle are the draft model's distribution q.

The draft proposes a token from q, and we accept it with probability the minimum of one and p over q. So the accepted mass at each token is the minimum of p and q: the green part of each middle bar. The red part on top is rejected.

The rest of the narration is in the book's Companion.