Latent

Watch a network reshape space

Push a grid and a dataset through a tiny network, layer by layer, and see linear maps stretch the plane while activations bend it.

IntermediateExplained in Why nonlinearity matters

Watch a network reshape space

A network with two neurons per layer, so every step is a 2D picture. Gridlines show what happens to the whole plane.

Input
Data
Activation
ShowingInput
Training accuracy100%
Linear stages can only rotate, stretch, shear, and shift the grid: straight lines stay straight. Activation stages bend it.

Try this

  • Play XOR with Tanh, then with None. Without an activation the classes never come apart.
  • Try ReLU and look for regions folded flat against an axis.
  • Drag the stage slider slowly through the activation stages to see the gridlines bend.

Try "embedding", "softmax", "overfitting", or "backpropagation".