Build an Autograd Engine

You computed one gradient by hand and gradient-checked it. Real networks have millions of parameters, so you cannot hand-derive each one. The trick that makes deep learning possible is automatic differentiation: every value remembers how it was computed, and gradients flow backward through that record automatically. You are about to build the engine, the same idea inside PyTorch and Karpathy's micrograd, in a few lines.

The unit is a Value: a number that also stores its gradient and a tiny function that knows how to push gradient to its inputs. When you add or multiply two Values, the result records its parents. The scaffold below gives you __init__ and a backward() that walks the graph in the right order (a topological sort, then chain-rule from the output back). You implement the two local gradients, which is the whole lesson:

def __add__(self, other):
    out = Value(self.data + other.data, (self, other))
    def _backward():
        self.grad  += out.grad      # d(a+b)/da = 1
        other.grad += out.grad      # d(a+b)/db = 1
    out._backward = _backward
    return out

Addition just copies the incoming gradient to both inputs. Multiplication is where the chain rule bites: d(a*b)/da = b and d(a*b)/db = a, each times the gradient flowing in from above (out.grad). Note +=, not =: a value used in several places accumulates gradient from each.

Complete __add__ and __mul__ so their _backward closures set the correct local gradients. The hidden tests build expressions, call .backward(), and gradient-check every result against a finite difference, so a wrong rule cannot pass. Press Run to backprop through a*b + a.