Latent

Overfitting

Raise a polynomial's degree until it memorizes the noise, watch test error climb, then rein it in with regularization.

BeginnerExplained in Overfitting and generalization

Overfitting

A polynomial is fit to 15 noisy training points. 200 more points from the same source are held back for testing.

  • Training points
  • Test points
  • Fitted degree 3 polynomial

About right: it follows the real trend without chasing every wiggle.

3
1e-3
Training error (MSE)0.0454
Test error (MSE)0.063
Best degree on test data3

Try this

  • Sweep the degree from 0 to 14 and find the bottom of the dashed test error curve.
  • At degree 14, turn on L2 regularization, then push the penalty strength to its maximum.
  • Press New sample of data at degree 3 and at degree 12, and compare how much the fit changes.

Try "embedding", "softmax", "overfitting", or "backpropagation".