Gaussian Process

A method that puts a probability distribution directly over functions, giving a prediction with uncertainty at every point.

A Gaussian process is a stochastic process: for any finite set of input points, the corresponding function values follow a joint (multivariate) normal distribution. It's fully specified by a mean function and a covariance function, also called a kernel, which describes how correlated the function values at two input points are. Used for regression, it lets you compute a predicted mean and variance at any new input given a small number of observations, giving uncertainty for free. The standard reference is Rasmussen and Williams's 2006 textbook 'Gaussian Processes for Machine Learning.' Its drawback is that computation scales as n³ with the number of data points n, so sparse approximations are needed once data gets large. In robotics it's mostly used where data is scarce: learning a dynamics model (such as Deisenroth and Rasmussen's 2011 PILCO, which learns control from scratch in just a handful of trials), or tuning control parameters with Bayesian optimization.

ExamplePILCO uses a Gaussian process to learn the dynamics of a cart-pole, folding the model's uncertainty into its long-horizon predictions, and learns to control it using only a handful of real trials, where ordinary reinforcement learning often needs hundreds or thousands.

Also called

GP, Gaussian Process Regression, GPR, Kriging

Related

System Identification · Model-Based Reinforcement Learning · Uncertainty Estimation · Sample Efficiency · Safe Reinforcement Learning · Kalman Filter

Sources

Gaussian process - Wikipedia
PILCO: A Model-Based and Data-Efficient Approach to Policy Search (Deisenroth & Rasmussen, ICML 2011)