Rearrangement

An embodied task where a robot moves objects in an environment to match a specified goal state.

Rearrangement is a unified task framework proposed in 2020 by Dhruv Batra, Sergey Levine, Jitendra Malik, and more than a dozen other researchers in “Rearrangement: A Challenge for Embodied AI.” Given a physical environment, the agent must move objects, open and close doors and drawers, and so on, to bring the environment to a specified goal state. The goal can be given as object poses, a goal image, or natural language, or the agent may first be shown the goal state directly. The framework packages navigation, perception, pick-and-place, and long-horizon planning into a single measurable task: tidying a room, setting a table, or restocking a shelf can all be written as rearrangement problems. Platforms such as AI2-THOR and Habitat have released corresponding benchmarks.

ExampleAI2-THOR's Visual Room Rearrangement (CVPR 2021): an agent first walks through a room to memorize where objects sit; afterward some objects are moved or have their open/closed state changed, and it must restore everything to how it was. The accompanying RoomR dataset covers 120 scenes, 72 object categories, and 6,000 configurations.

Also called

Object Rearrangement, Scene Rearrangement

Related

Household Tasks · Mobile Manipulation · Long-horizon Task · Pick-and-Place · Embodied Agent · Habitat

Sources

Rearrangement: A Challenge for Embodied AI (arXiv 2011.01975)
Visual Room Rearrangement (CVPR 2021)