Uncertainty Estimation

Having a model report how confident it is alongside its prediction, so it's clear when to stop or ask for help.

Uncertainty estimation studies how to quantify how confident a model is in its own output. It's usually split into two kinds: aleatoric uncertainty comes from randomness in the data itself, such as sensor noise, and doesn't go away with more data; epistemic uncertainty comes from the model not having seen or learned something well, and can be reduced with more data. Common methods include deep ensembles (training several models and looking at how much they disagree), Monte Carlo dropout (randomly dropping units multiple times at inference and looking at the spread of results), and conformal prediction (producing a candidate set with a statistical coverage guarantee). For robots, this matters directly for safety: when facing an out-of-distribution scene, high uncertainty can trigger slowing down, stopping, asking a person to take over, or actively collecting more data; model-based reinforcement learning also often treats disagreement across an ensemble of models as an exploration signal.

ExampleKnowNo (CoRL 2023) uses conformal prediction to measure a large-language-model planner's uncertainty: when the candidate-action set has more than one option — say, two bowls are on the table and the instruction doesn't say which one to pick up — the robot proactively asks a person, keeping task success guaranteed while asking for help as rarely as possible.

Also called

Uncertainty Quantification, UQ

Related

Out-of-Distribution · Robustness · Embodied Safety · Failure Recovery · Human-in-the-Loop · KnowNo

Sources

Wikipedia: Uncertainty quantification
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles (arXiv:1612.01474)
Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners (arXiv:2307.01928)