Human-in-the-Loop

Keeping a person inside a robot's training or operating loop to correct, take over, or give feedback in real time.

Human-in-the-loop broadly describes a system that needs a person involved while it runs: taking over control when the robot errs, providing a corrective action, scoring an outcome, or confirming a critical decision. In robot learning, its main job is covering the gap left by offline imitation learning: once a policy drifts into a state the demonstrations never covered, it tends to get further and further off track (compounding error), and human corrections at exactly those states supply the missing data. Notable methods include DAgger and 2018's HG-DAgger (a human takes over when they judge things are about to go wrong, and the segments they took over become new training data), and UC Berkeley's HIL-SERL (2024), where a person intervenes on real-robot reinforcement learning at any time using a 3D mouse, letting the robot learn precision assembly, bimanual coordination, and similar tasks in 1 to 2.5 hours. Human takeover during deployment and remote-teleoperation fallback also fall under this umbrella, and the intervention rate is a common way to measure how autonomous a system really is.

ExampleDuring HIL-SERL training, an operator holds a SpaceMouse and watches over the robot, taking over just before it's about to fail; that intervention data goes into both the demonstration buffer and the reinforcement-learning buffer, speeding up learning.

Also called

HITL, Human Intervention

Related

DAgger · Human-Gated DAgger · HIL-SERL · Human Intervention Data · Intervention Rate · Shared Autonomy

Sources

Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning (HIL-SERL)
HG-DAgger: Interactive Imitation Learning with Human Experts
Wikipedia: Human-in-the-loop