Backpropagation, step by step

Backpropagation, one node at a time

A single tanh neuron with squared-error loss. Run the forward pass, then send gradients backward with the chain rule.

Press Next step to run the forward pass: each node computes its value from its inputs.

Each box shows a value and, once the backward pass reaches it, its gradient ∂L/∂(that value) after the ∂ sign. Inputs x₁ = 1.000, x₂ = −0.500, target y = 1.000. Highlighted boxes are the parameters the network can change.

Weight w₁0.40

Weight w₂0.80

Bias b−0.30

Step0 of 15

Loss Lnot computed yet

Try this

Step through the forward pass, then the backward pass, checking each product against the explanation.
After the backward pass, press Update weights and step through again. The loss is smaller.
Use the sliders to make the loss small, then run the backward pass again. When the output is close to the target, every gradient is small too, because each one carries the factor $2(a - y)$ .