Latent

Confident, and wrong

A model trained on one region of data is tested somewhere else. Its accuracy collapses while its confidence stays high.

NoviceExplained in What AI gets wrong

Confident, and wrong

A single neuron learned from data in the dashed box on the left. Slide the solid box to test it on new data from elsewhere.

Inside the training area
Accuracy where it learned99%
Accuracy on the new data100%
How sure it sounds on the new data97%
The solid line is the model's boundary; the dashed line is the true rule. Ringed points are new examples the model gets wrong. "How sure it sounds" averages the model's probability for whichever answer it gives, from 50% (a coin flip) to 100%.

Try this

  • Slide the new data from the left edge to the right edge and watch the two accuracy numbers part ways.
  • Turn on Show the true rule to see why: the truth curves, but the model only ever saw the straight part.
  • Watch the confidence number as accuracy falls. Does the model warn you?

Try "embedding", "softmax", "overfitting", or "backpropagation".