Mana: Dexterous Manipulation of Articulated Tools
2026-06-11 • Robotics
RoboticsArtificial IntelligenceComputer Vision and Pattern RecognitionMachine Learning
AI summaryⓘ
The authors developed Mana, a system that treats the tricky problem of using tools with moving parts like making an animation. Instead of just focusing on rigid tools, their method turns simple grasp positions into detailed hand movements automatically using planning and learning techniques. This process mostly runs on its own and only needs a few clicks to set up for each tool. They tested Mana on different tools with joints and showed it works well in the real world without extra training. This offers a new way to teach robots how to skillfully use complex tools.
articulated toolsdexterous manipulationsim-to-real transfermotion planningreinforcement learningcomputer animationgraspingmanipulation trajectoriesfunctional affordancesrobotics
Authors
Zhao-Heng Yin, Guanya Shi, Pieter Abbeel, C. Karen Liu
Abstract
Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact-rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies. We present Mana (Manipulation Animator), a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally-generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning. The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances (<1 minute per tool). Across four articulated tools spanning different scales and joint types, Mana achieves zero-shot sim-to-real transfer for both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.