A Neuron from Scratch

All of deep learning is built from one tiny part: the artificial neuron. It does three things, in order.

• Weighted sum. Multiply each input by its own weight and add them up. A weight says how much that input matters.
• Add a bias. A single number that shifts the result up or down, like an intercept.
• Activation. Squash the result through a nonlinear function so the neuron can express more than a straight line.

A classic activation is the sigmoid, which maps any number to a value between 0 and 1 (handy for probabilities):

sigmoid(z) = 1 / (1 + e^(-z))

Some landmarks: sigmoid(0) is exactly 0.5, big positive inputs approach 1, and big negative inputs approach 0. Put it together and a neuron is just:

import numpy as np

def neuron(x, w, b):
    z = np.dot(x, w) + b      # weighted sum + bias
    return 1 / (1 + np.exp(-z))  # sigmoid activation

np.dot(x, w) does the multiply-and-add for the whole input vector at once. That is the entire idea. Stack thousands of these and train the weights, and you have a neural network.