Egocentric Video

Video filmed from a head- or glasses-mounted camera, showing the wearer's own point of view with the hands in frame.

Egocentric video is filmed by a camera worn on the head or on a pair of glasses, so the frame looks like what the wearer's own eyes would see, with the hands and whatever they're handling usually near the center — the counterpart to third-person (exocentric) video. It's useful for robotics because the viewpoint is close to a robot's head-mounted camera and shows hand-manipulation detail clearly. Notable datasets include Ego4D (released 2022, led by Meta with 13 universities, covering more than 3,670 hours of daily activity) and Apple's EgoDex, captured with the Vision Pro and annotated with 3D hand-joint positions (829 hours). This kind of video carries no robot action labels, so it first has to be converted — through hand-pose estimation, or through latent-action methods that infer an abstract action from a pair of consecutive frames — before it can be used to train a policy.

ExampleEgoDex records 194 kinds of tabletop manipulation tasks with an Apple Vision Pro, saving the 3D position of every finger joint at the same time, so the recording can be used directly as a hand-motion trajectory.

Also called

Ego, First-person Video, Egocentric View

Related

Exocentric Video · Human Video Data · Ego4D · EgoDex · Project Aria Glasses · Hand Pose Estimation

Sources

Ego4D 官网 (Chinese)
EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video (arXiv 2505.11709)

As of

2025-05