Energy-Based Model

A model that scores input-output pairs instead of predicting the output directly; the lowest-scoring output is the answer.

An energy-based model doesn't predict its output directly; instead, it learns a scoring function E(x, y) — the better x and y match, the lower the 'energy' — and inference means searching for the y with the lowest energy. The best-known robotics application is Google's 2021 Implicit Behavioral Cloning (IBC): a network assigns an energy score to an 'observation, candidate action' pair, then derivative-free optimization or Langevin sampling (noisy gradient descent) is used to find the lowest-scoring action. This lets a single observation map to several valid actions and lets the policy represent discontinuities, and the paper reached about 1 mm of precision on contact-rich real-robot tasks. The cost is that training needs sampled negative examples to approximate the normalization constant, and the Diffusion Policy paper notes that IBC training is unstable and hard to tune. Since then, modeling multimodal actions has mostly shifted to Diffusion Policy and flow matching instead.

ExampleOn the same tabletop image, pushing a block from the left or from the right both complete the task. An energy-based model assigns low energy to both actions and lands on one of them at inference; a mean-squared-error regression model instead outputs their average, which is wrong either way.

Also called

EBM, Implicit Behavioral Cloning, IBC

Related

Behavior Cloning · Action Multimodality · Diffusion Policy · Mixture Density Network · InfoNCE Loss · Generative Model

Sources

Implicit Behavioral Cloning (Florence et al., arXiv 2109.00137)
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (arXiv 2303.04137)
Energy-based model - Wikipedia