RFM-1

Covariant's roughly 8-billion-parameter robot foundation model, trained on warehouse picking data.

RFM-1 is a robot foundation model released by warehouse-robotics AI company Covariant on March 11, 2024, with about 8 billion parameters. Its training data comes from Covariant's own picking arms deployed in warehouses across 15 countries, reportedly amounting to tens of millions of trajectories, including images, video, joint angles, and readings such as force and suction-cup vacuum level, plus text. Both its inputs and outputs can be text, images, video, or robot actions: it can understand natural-language instructions, and it can also predict what the scene will look like after a given action, effectively acting as a learned physics simulator to anticipate the consequences of an action. Its limitation is that it mainly covers vacuum-suction warehouse picking, with limited generalization to entirely new objects and situations. In August 2024, Amazon obtained a non-exclusive license to Covariant's technology and brought on founders Pieter Abbeel, Peter Chen, and Rocky Duan.

ExampleAn operator can tell a picking arm in natural language to 'pick up the red one'; RFM-1 can also generate a predicted image of the scene after executing a given grasp, letting the outcome be anticipated in advance.

Also called

Robotics Foundation Model 1, Covariant RFM-1

Related

Foundation Model · World Model · Bin Picking · Covariant · Vacuum Suction Cup · Amazon Frontier AI & Robotics

Sources

IEEE Spectrum: Covariant Announces a Universal AI Platform for Robots
TechCrunch: Covariant is building ChatGPT for robots
Wikipedia: Covariant (company)

As of

2024-08