Watch: Backprop & the Chain Rule

Gradient descent needs one thing: the gradient of the loss with respect to every input and weight. Backpropagation is how you get it, and it is just the chain rule applied to a graph of operations.

Press play. Push a value forward through a tiny graph (x -> double -> square -> L), then watch the gradient flow backward, multiplying by each step's local gradient. Once you see it, the autograd engine you build next is no longer magic.