The interactive film is in the book's Companion

Play it, pause it, turn it and drag it with Mathematics for AI and Machine Learning.

Get the book

From a Residual Network to a Neural ODE

Chapter 20 · Mathematics for AI and Machine Learning · Beginner · 1:28

An interactive live film: a ring of features rises through the layers of a residual network, x + (1/L) f(x) per layer; with more layers the jagged paths become the smooth curves of the neural ODE dx/ds = f(x), and the gap shrinks like 1/L.

Narration

A residual network updates its features one layer at a time: x next equals x plus one over L times f of x. The floor is the feature plane, and this f turns every point and pulls it inward.

Stack the layers upward, with depth running from zero to one. With four layers, each point takes four big straight steps, and its path is jagged.

The rest of the narration is in the book's Companion.

More from Chapter 20

Interactive1:45Flow Matching: Velocity Fields and the Continuity EquationChapter 20Preview1:27Gradient Flow: Gradient Descent With Infinitely Small StepsChapter 20Preview1:26Memory by Orthogonal Projection (HiPPO)Chapter 20Preview1:36Momentum Methods: Second-Order ODEChapter 20Preview1:34Probability Flow ODE: Deterministic Paths, Identical MarginalsChapter 20Preview1:40Backward Error Analysis: What Gradient Descent FollowsChapter 20Preview1:46Rectified Flow: Straightening by RetrainingChapter 20