Sensorimotor Skills

The ability to turn sensory information into body movement in real time: grasping, walking, catching, twisting.

Sensorimotor skills is a term from cognitive science and neuroscience for abilities that tightly couple sensory input, such as vision, touch, and proprioception, with muscle movement to complete a task: reaching for an object, walking while keeping balance, catching a thrown ball. In robotics it broadly refers to low-level capabilities that need perception and control to run in a real-time closed loop. Moravec's paradox observes that the sensorimotor skills people find effortless are actually the hardest for machines, precisely because they were refined over a long evolutionary history. Levine and colleagues' 2015 end-to-end visuomotor policy work, which used a convolutional network to map images directly to motor torques, is a landmark example of learning such skills with deep learning; today's visuomotor policies and VLA models are, at bottom, also learning sensorimotor skills.

ExampleUnscrewing a bottle cap: a robot must watch the position of the cap while continuously adjusting the angle and force it applies based on what it feels in its hand — this was one of the test tasks in Levine and colleagues' end-to-end visuomotor policy work.

Also called

Sensorimotor Skill Learning

Related

Moravec's Paradox · Perception-Action Loop · Visuomotor Policy · Embodied Cognition · End-to-End Training of Deep Visuomotor Policies

Sources

Moravec's paradox - Wikipedia
End-to-End Training of Deep Visuomotor Policies (arXiv 1504.00702)
Sensory-motor coupling - Wikipedia