A Two-Layer Forward Pass

One neuron is a line. Stack a layer of them, add a nonlinearity, then feed that into another layer, and the network can bend to fit almost anything. Running data through the layers to get an output is called the forward pass.

A layer is just a matrix multiply: each column of the weight matrix is one neuron's weights. With numpy, the whole layer is x @ W + b (the @ operator is matrix multiplication).

Between layers we apply a nonlinear activation. Modern nets favour ReLU (Rectified Linear Unit), which is simply max(0, z): keep positives, zero out negatives. Without a nonlinearity, stacking layers would collapse back into a single line.

import numpy as np

def relu(z):
    return np.maximum(0, z)

h = relu(x @ W1 + b1)   # hidden layer
out = h @ W2 + b2       # output layer

Why we do this in numpy here

The real frameworks for this are TensorFlow and PyTorch. They cannot run in this lesson: Pyodide is CPython compiled to WebAssembly in your browser, and those libraries ship heavy native C++ and CUDA/GPU code that has no WebAssembly build. So we get the intuition with a few lines of numpy, which is exactly what a forward pass is under the hood. The frameworks add autograd, GPUs, and scale, not new ideas about this step.

Watch the shapes: a length-3 input through a 3x4 weight matrix gives 4 hidden values; through a 4x1 matrix that becomes a single output.