Early Stopping

Stopping training once validation performance stops improving, and keeping the checkpoint that performed best.

Early stopping is one of the most common regularization techniques. During training, the model is periodically evaluated on a validation set (data held out from training, used specifically to pick a model): once training loss keeps falling but validation loss starts rising, the model has begun to overfit, so training stops and the checkpoint with the best validation performance is kept instead of the final one. In practice, a “patience” value is often set — stop only after several rounds in a row with no improvement — to avoid being misled by noise. In robot imitation learning this needs extra care: low validation loss doesn't guarantee a high real-robot success rate. robomimic's research found the training objective and the evaluation objective don't always agree, so which step training stops at matters a great deal, and many projects save several checkpoints and pick among them using simulation or real-robot rollouts instead. Don't confuse this with “early termination” in reinforcement learning, which ends a single episode early, for instance when the robot falls.

ExampleSay a grasping policy is trained with validation action error computed once per epoch: the error is lowest at epoch 30 and doesn't improve for 10 epochs after that, so training stops and the epoch-30 checkpoint is kept.

Related

Overfitting · Training / Validation / Test Set · Checkpoint · Regularization · Epoch · Early Termination

Sources

Wikipedia: Early stopping
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation (robomimic)
Google Machine Learning Glossary