HumDex:Humanoid Dexterous Manipulation Made Easy
2026-03-12 • Robotics
Robotics
AI summaryⓘ
The authors created HumDex, a portable system that helps control humanoid robots' whole bodies more precisely by using motion sensors worn on the human operator. They also developed a learned method to make robot hand movements smooth without needing complicated adjustments. Using HumDex, they gathered human motion data to train robots first on general human movements and then fine-tune for robot-specific actions, improving the robot’s ability to handle new tasks with less training data. Their system is open-source and designed to be easy to use for complex robot manipulation.
humanoid robotsteleoperationIMU motion trackingdexterous manipulationimitation learningmotion retargetingrobot controldata efficiencyopen-source robotics
Authors
Liang Heng, Yihe Tang, Jiajun Xu, Henghui Bao, Di Huang, Yue Wang
Abstract
This paper investigates humanoid whole-body dexterous manipulation, where the efficient collection of high-quality demonstration data remains a central bottleneck. Existing teleoperation systems often suffer from limited portability, occlusion, or insufficient precision, which hinders their applicability to complex whole-body tasks. To address these challenges, we introduce HumDex, a portable teleoperation system designed for humanoid whole-body dexterous manipulation. Our system leverages IMU-based motion tracking to address the portability-precision trade-off, enabling accurate full-body tracking while remaining easy to deploy. For dexterous hand control, we further introduce a learning-based retargeting method that generates smooth and natural hand motions without manual parameter tuning. Beyond teleoperation, HumDex enables efficient collection of human motion data. Building on this capability, we propose a two-stage imitation learning framework that first pre-trains on diverse human motion data to learn generalizable priors, and then fine-tunes on robot data to bridge the embodiment gap for precise execution. We demonstrate that this approach significantly improves generalization to new configurations, objects, and backgrounds with minimal data acquisition costs. The entire system is fully reproducible and open-sourced at https://github.com/physical-superintelligence-lab/HumDex.