SpaceMouse Teleoperation

Controlling a robot arm's end effector in six degrees of freedom by pushing, pulling, twisting, and tilting a 3D mouse.

The SpaceMouse is a 6-DoF input device from 3Dconnexion, originally built for panning, zooming, and rotating 3D models in CAD software; its knob can be pushed sideways, pressed up and down, twisted, and tilted. When used for teleoperation, these 6 quantities are mapped onto incremental translation and rotation of the robot arm's end effector, with a button controlling the gripper's open/close. It's cheap and plug-and-play, needing no leader-follower arm or VR headset, which makes it well suited to single-arm tabletop tasks; simulation frameworks like robosuite support it natively. The downside is that it only controls end-effector pose, so it doesn't suit bimanual or dexterous-hand setups, feels less intuitive, and produces slower, less natural motion than a leader-follower arm. In human-in-the-loop reinforcement learning, it's commonly used as the tool a human uses to take over and correct the robot at any moment.

ExampleDuring HIL-SERL training, a person holds a SpaceMouse while supervising the robot; whenever the policy makes a mistake, the person takes over directly through the SpaceMouse and provides a corrective action, and these intervention episodes are stored in the replay buffer to continue training.

Also called

SpaceMouse, 3D Mouse

Related

Teleoperation · Human-in-the-Loop · HIL-SERL · Human Intervention Data · robosuite · Leader-Follower Teleoperation

Sources

robosuite Docs: I/O Devices (3Dconnexion SpaceMouse)
Precise and Dexterous Robotic Manipulation via Human-in-the-Loop RL (HIL-SERL, arXiv 2410.21845)
Wikipedia: 3Dconnexion