Optimizer race
Four optimizers start from the same point on the same surface. Tap the map to move the start.
The Rosenbrock function: easy to find the valley, slow to follow its gently curving floor to the minimum.
Tap to move the start
- Gradient descent
- Momentum
- RMSProp
- Adam
Steps0
Gradient descent7.016
Momentum7.016
RMSProp7.016
Adam7.016
Try this
- Race on the Banana valley with Fast forward, then scale the step sizes to 4x.
- On the Saddle point, count how many steps each optimizer needs to slide off the ridge.