The artificial neuron

You have seen models that vote among neighbors and models that fit a line. The models behind image recognition and chatbots are neural networks, and they are built from millions or billions of copies of one small calculation: the Artificial neuronThe basic unit of a neural network. It multiplies each input by a weight, adds them up with a bias, and passes the total through an activation function.Open in glossary. Understand one, and you understand the brick the whole building is made of.

A decision, written as arithmetic

Suppose you are deciding whether today is a good day for a bike ride. Two things matter most: how warm it is and how likely it is to rain. Warmth makes you more keen; rain makes you less keen. And you might be the kind of person who needs a lot of convincing, or the kind who goes out in almost anything.

A neuron turns exactly that kind of reasoning into arithmetic:

Inputs. Each fact arrives as a number. Here, warmth and rain risk, each on a scale from 0 to 10.
WeightA number that sets how much one input matters to a neuron, and in which direction. Positive weights push the output up, negative weights push it down.Open in glossary. Each input is multiplied by a weight that says how much it matters and in which direction. A positive weight pushes toward yes; a negative weight pushes toward no.
Bias (neuron)A number a neuron adds to its weighted sum before the activation function. It shifts how easily the neuron switches on, regardless of the inputs.Open in glossary. The weighted inputs are added up, plus one more number, the bias. It shifts the total up or down whatever the inputs are, setting how easily the neuron says yes.
Activation. The total can be any number, large or small. An Activation functionThe function a neuron applies to its weighted sum, such as the sigmoid, tanh, or ReLU. Without it, stacking layers of neurons would add no power.Open in glossary squashes it into a range that is easy to use. Here it turns the total into a chance between 0% and 100%.

Below, the diagram shows that calculation for one day, marked Today. The plane shows the neuron’s answer for every possible day at once: deeper orange means more confident it is a good ride, deeper blue more confident it is not. The triangles and circles are thirty past days, labeled by whether they turned out good for a ride.

One artificial neuron

Should you go for a bike ride? The neuron weighs warmth and rain, adds a bias, and squashes the total into a chance from 0% to 100%.

Rain risk

Warmth

Weight on warmth0.60

Weight on rain0.30

Bias−3.0

Chance today is a good ride89%

Past days on the correct side16 of 30

Loss1.063

Triangles are days that were good for a ride, circles days that were not. Tap the plane to move Today.

Try this

Look at the weight on rain. It starts positive, which means the neuron thinks rain makes a ride more appealing. Drag it below zero and watch the boundary swing around.
Move the bias left and right. The boundary slides without turning. A lower bias means the inputs have to add up to more before the neuron says yes.
Double both weights, keeping their signs. The line keeps its angle (it also slides, because the bias did not double with them), and the shading gets sharper: the neuron becomes more decisive.
Tap the plane to move Today and watch the numbers in the diagram change.
Press Learn from the examples and watch the neuron find its own weights from the past days.

A neuron draws a straight line

The dark line on the plane is where the neuron says exactly 50%. On one side it leans yes, on the other it leans no. That line is called the Decision boundaryThe line or surface where a classifier switches from predicting one class to another.Open in glossary, and for a single neuron it is always straight, no matter how you set the weights.

Here is why. The neuron sits at 50% exactly when the total before squashing is zero: weight times warmth, plus weight times rain, plus bias, equals zero. Every combination of warmth and rain that satisfies an equation like that falls on one straight line. The weights set the line’s angle; the bias slides it.

A straight line is enough for the bike ride. It is not enough for many real problems, where the examples of one kind might sit in a circle surrounded by the other kind. The next lesson shows how combining neurons gets around that.

Learning means finding the weights

When you pressed Learn from the examples, the neuron did what the line-fitting computer did in the last lesson. It measured how wrong its answers were on the past days, worked out which way to nudge each weight and the bias to be less wrong, nudged, and repeated. The weights it settled on are its learned knowledge: everything it “knows” about bike rides is in those three numbers.

The neuron as a formulaOptional

With inputs $x_1, x_2$ , weights $w_1, w_2$ , and bias $b$ , the neuron computes a weighted sum

z = w_1 x_1 + w_2 x_2 + b

and passes it through the SigmoidThe S-shaped function 1 / (1 + e^-z), which squashes any number into the range 0 to 1 so it can be read as a probability.Open in glossary function

\sigma(z) = \frac{1}{1 + e^{-z}},

which is 0.5 when $z = 0$ , approaches 1 for large positive $z$ , and approaches 0 for large negative $z$ . The decision boundary is the line $w_1 x_1 + w_2 x_2 + b = 0$ .

A single neuron with a sigmoid output is the same model statisticians call logistic regression. Learning uses the loss called cross-entropy, which punishes confident wrong answers heavily, and the demo improves it with gradient descent (specifically the Adam variant, which picks a good step size for each weight on its own).

Key ideas

An artificial neuron multiplies each input by a weight, adds a bias, and passes the total through an activation function.
A weight’s size says how much an input matters; its sign says which way it pushes.
The bias shifts how easily the neuron says yes.
A single neuron’s decision boundary is always a straight line.
Learning means adjusting the weights and bias to reduce the loss on examples.