Era of Experience

Silver and Sutton's view that AI's next stage will learn mainly from its own experience of interacting with the world.

“Era of Experience” comes from a short paper published by David Silver and Richard Sutton in April 2025, “Welcome to the Era of Experience,” a preprint of one chapter from the MIT Press book Designing an Intelligence; Silver was one of the lead figures behind the AlphaGo line of work, and Sutton is one of the founders of reinforcement learning. The paper argues that the “era of human data,” training on massive amounts of human-generated data and then fine-tuning on human preferences, is nearing its ceiling in areas such as mathematics, coding, and science, and that the next stage of agents should learn mainly from experience generated by their own interaction with the environment. It lists four defining features: living in a long stream of experience rather than short conversations; actions and observations grounded in the environment; rewards coming from real signals in the environment rather than being judged by people in advance; and planning and reasoning also grounded in experience. For embodied AI, this view supports the idea of robots continuously trying, failing, and improving themselves through real-world deployment.

ExampleThe paper cites AlphaProof as an example: it first studies roughly 100,000 human-written formal proofs, then generates about 100 million more on its own by interacting with a proof-checking system, eventually reaching medal-level performance at the International Mathematical Olympiad.

Also called

Welcome to the Era of Experience

Related

Reinforcement Learning · The Bitter Lesson · Real-World Reinforcement Learning · Self-improvement · Data Flywheel · Continual Learning

Sources

Welcome to the Era of Experience (Silver & Sutton, 2025, preprint)

As of

2025-04