Compounding Error

Small per-step deviations in an imitation-learned policy add up, pushing the robot into unfamiliar states where it fails.

Compounding error is the central problem with behavior cloning (imitating an expert's actions directly via supervised learning). The policy makes a small error at every step, and once it drifts off the expert's trajectory, it ends up in states that never appeared in the training data, where it's even more likely to err — the deviation snowballs. Ross and Bagnell's 2010 analysis showed that with a per-step error rate of ε over a task of length T, pure supervised imitation can incur cost as bad as order T²ε relative to the expert, growing quadratically with task length. The underlying cause is covariate shift. Common mitigations include DAgger, which has the expert label the states the policy actually wanders into; deliberately collecting recovery data for correcting mistakes; and action chunking, as in ACT, which predicts a whole block of actions at once and reduces the number of decision points.

ExampleThe ACT paper notes that in high-precision bimanual fine manipulation, an imitation-learned policy's error accumulates over time, so instead it predicts an entire action sequence at once, reaching 80–90% success on several tasks using just 10 minutes of human demonstrations.

Also called

Error Compounding, Error Accumulation, Compounding Errors

Related

Distribution Shift · DAgger · Behavior Cloning · Action Chunking · Recovery and Correction Data · Exposure Bias

Sources

Efficient Reductions for Imitation Learning (Ross & Bagnell, AISTATS 2010)
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (AISTATS 2011)
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (ACT, arXiv 2304.13705)