A layer is a matrix multiplication
Each output neuron takes a weighted sum of the inputs plus a bias. Stack those weights in rows and you have a matrix.
positive weightnegative weightThicker means larger.
W
×x
+1.00.5−1.0
b
=0.20.00.30.4
z
0.30−0.800.501.35
z₁ = (0.7)(1.0) + (−1.4)(0.5) + (−0.1)(−1.0) + 0.2 = 0.30
ShapesW is 4 × 3, x has 3, b and z have 4
Neuron 1 output z₁0.30
Try this
- Select an output neuron and find its row of W. The highlighted lines and the highlighted row are the same weights.
- Set all inputs to 0: every output becomes its bias.
- Press New weights and confirm the shapes never change.