Sampling-based MPC

MPC that samples many candidate action sequences each cycle, scores them by simulation, and executes the best one's first step.

Model predictive control (MPC) optimizes a short sequence of future actions every cycle but executes only the first step before replanning. Traditional solvers rely on gradients or quadratic programming, which requires a differentiable, smooth cost function. Sampling-based MPC instead samples tens to thousands of candidate action sequences by adding noise around the previous optimal sequence, rolls them out in parallel through a dynamics model (often a physics simulator) to score each one, and then picks the best or takes a cost-weighted average to form the new sequence. Representative methods include MPPI (Model Predictive Path Integral control), which weights samples proportional to exp(−cost/λ) with temperature λ; the cross-entropy method (CEM), which keeps the lowest-cost batch and refits the sampling distribution; and Predictive Sampling, the simplest version, released by DeepMind alongside MuJoCo MPC in 2022. These methods need no derivatives, can handle non-smooth dynamics such as contact, and parallelize well on GPUs — though sampling efficiency drops as the action dimension grows.

ExampleDIAL-MPC, proposed by CMU and others in 2024, borrows the step-by-step denoising and annealing idea from diffusion models to do force-level sampling optimization directly on full quadruped dynamics without any training. The paper reports tracking error 13.4 times lower than standard MPPI, and demonstrated precise loaded jumps on a real robot.

Also called

Sampling-Based Model Predictive Control

Related

Model Predictive Control · Model Predictive Path Integral Control · Cross-Entropy Method · DIAL-MPC · MuJoCo MPC (MJPC) · Trajectory Optimization

Sources

Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo (Howell et al., arXiv 2212.00541)
Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving (Williams et al., arXiv 1707.02342)
Full-Order Sampling-Based MPC for Torque-Level Locomotion Control via Diffusion-Style Annealing (DIAL-MPC, arXiv 2409.15610)