Score Function / Score Matching

The 'score' is the gradient of log-probability with respect to the input; score matching is how a network learns it.

The score function is the gradient of the log probability density with respect to the input, ∇ₓ log p(x); it points in the direction where the data's probability increases fastest, and computing it doesn't require the hard-to-compute normalization constant. Score matching was proposed by Hyvärinen in 2005 as a way to train a model to fit this gradient without knowing the true distribution; the later denoising score matching variant instead 'adds noise to the data, then learns how to remove it.' In 2019, Song and Ermon learned the score across many noise levels and generated images with Langevin dynamics (a sampling method that follows the gradient while adding a bit of random noise at every step); the 2021 stochastic differential equation (SDE) framework then showed this is essentially the same class of model as denoising diffusion probabilistic models. What a diffusion model predicts as 'noise' is the score up to a rescaling, so understanding the score is the key to understanding what Diffusion Policy is really doing underneath.

ExampleThe Diffusion Policy paper describes its own method as learning a score gradient field over the action distribution: at inference, starting from random noise, it takes several noisy, Langevin-like steps along this gradient field and ends up with a robot action.

Also called

Score, Denoising Score Matching, Score-based Generative Model

Related

Diffusion Model · Denoising Diffusion Probabilistic Model · Energy-Based Model · Diffusion Policy · Flow Matching · Diffusion / Flow Samplers

Sources

Yang Song: Generative Modeling by Estimating Gradients of the Data Distribution (blog)
Generative Modeling by Estimating Gradients of the Data Distribution (arXiv:1907.05600)
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (arXiv:2303.04137)