How machines learn
The core ideas every model shares. Measure how wrong a model is, follow the slope downhill to improve it, and learn why a model that memorizes its training data can still fail on new data.
- Level
- Beginner
- Length
- 8 lessons, 2 hr 8 min
- Assumes
- Comfort with graphs and basic algebra.
- Builds on
- AI from zero
- 1The learning loopEvery trained model, from a straight line to a chatbot, improves by repeating the same four steps. Run the loop yourself and watch a line learn.14 min
- 2Linear regression and lossWhat makes one line fit better than another? Measure the misses, find the best line by hand, and see how the choice of loss decides which line wins.15 min
- 3Gradient descentMost models have no formula for their best parameters. They find them by repeatedly stepping downhill on the loss. See how the slope guides each step, and how the step size can make or break training.18 min
- 4Better optimizersPlain gradient descent struggles in narrow valleys and on flat ground. Momentum, RMSProp, and Adam fix this with two simple ideas. Race them on the same surfaces.16 min
- 5Classification and probabilityTo sort things into categories, a model turns a score into a probability. Meet the sigmoid, the cross-entropy loss, and the decision boundary, and train a classifier live.17 min
- 6Overfitting and generalizationA model flexible enough to fit anything will happily fit the noise too. See training error and test error split apart, and learn the two standard fixes.17 min
- 7Measuring a classifierAccuracy can be badly misleading. Learn the confusion matrix, precision, recall, and the ROC curve by moving a spam filter's threshold yourself.16 min
- 8Finding groups: k-meansNot every dataset comes with answers. k-means finds groups in unlabeled data with two simple alternating steps. Watch it work, and watch it fail.15 min
Labs in this track
- Choosing a thresholdDrag a spam filter's threshold and watch the confusion matrix, precision, recall, and ROC curve respond.
- Fitting a lineDrag points, fit a line by hand, and compare the best lines under squared and absolute error when an outlier appears.
- Gradient descentDrop a starting point on a loss surface and watch gradient descent find its way down, or overshoot and diverge.
- k-means, step by stepWatch k-means find groups in unlabeled points by alternating assign and update steps, and see where it goes wrong.
- Logistic regressionTrain a classifier live and watch its decision boundary and probability shading settle between two classes.
- Optimizer raceRace plain gradient descent, momentum, RMSProp, and Adam from the same start on four loss surfaces.
- OverfittingRaise a polynomial's degree until it memorizes the noise, watch test error climb, then rein it in with regularization.
- The learning loopStep a line through predict, measure, find the direction, and update, and watch training and test loss fall.