Procedural Generation

Automatically producing large numbers of scenes, objects, or terrain from rules plus randomness, instead of building each one by hand.

Procedural generation originated in the game industry: content is produced automatically by an algorithm plus randomness, the way Minecraft generates an entire map from a single random seed. In embodied AI, it is used to mass-produce simulated training environments: rules are written first (how to partition a floor plan, where furniture goes, which materials and textures to use, how to light the scene), and then thousands of distinct scenes are randomly sampled from those rules. Building one interactive 3D house by hand is slow, and too few scenes make a policy prone to overfitting, so procedural generation trades rule-writing effort for quantity and diversity, making it a standard tool for improving generalization across scenes. Notable projects include the Allen Institute for AI's ProcTHOR and Princeton's Infinigen; the randomly generated stairs and slopes used in quadruped reinforcement learning also fall into this category. Its counterpart is generative simulation, which instead uses a large model to build scenes from text descriptions.

ExampleProcTHOR can randomly generate floor plans of 1 to 10 rooms, sampling and arranging from 108 categories, 1,633 interactive objects, and 3,278 materials; the paper pretrained an embodied agent on 10,000 generated houses, and without any fine-tuning on downstream data it often beat the previous best methods.

Also called

Procedural Scene Generation, Procedural Content Generation, PCG

Related

ProcTHOR (Large-Scale Embodied AI Using Procedural Generation) · Infinigen · Domain Randomization · Generative Simulation · Scene Generalization · Terrain Curriculum

Sources

Wikipedia: Procedural generation
ProcTHOR: Large-Scale Embodied AI Using Procedural Generation (arXiv 2206.06994)
Infinigen