PhysClaw-0: A Symbiotic Agentic System for Robot Autonomy via Language Corrections
2026-07-15 • Robotics
RoboticsHuman-Computer Interaction
AI summaryⓘ
The authors developed PhysClaw-0, a system that helps a robot learn how to manipulate objects by collecting data more efficiently. Instead of asking humans to fix every repeated mistake, their system remembers corrections and reuses them, which saves a lot of human effort. They use language processing to turn human feedback into structured fixes that the robot can apply automatically. Testing on a real robot showed their system worked as well as manual control but needed much less human time. Overall, their approach improves success rates and reduces the cost of teaching robots.
autonomous data collectionmanipulation policy learningself-resettinglarge language model (LLM)natural language processingcorrective memoryteleoperationrobotics manipulationverificationfine-tuning
Authors
Boyuan Wang, Zhenyuan Zhang, Zhiqin Yang, Peijun Gu, Shuya Wang, Xiaofeng Wang, Xianghui Ze, Yifan Chang, Guosheng Zhao, Jiangnan Shao, Guan Huang, Hengyu Liu, Yonggang Zhang, Wei Xue, Chunyuan Guan, Chenglin Pu, Yike Guo, Xingang Wang, Zheng Zhu
Abstract
Autonomous data collection governs the volume and quality of real-world trajectories for manipulation policy learning. Existing pipelines reduce human effort via self-resetting, VLM verification, or language-guided correction, yet episode-scoped fixes must be reissued whenever the same failure recurs, so oversight cost grows with session length rather than with the number of distinct problems. We present PhysClaw-0, a human-robot symbiotic agentic system in which corrections are retained and reused across rounds. The collection loop collects, verifies, and resets autonomously, pausing for a remote operator only when a phase exhausts an explicit retry budget. An LLM parser maps each natural-language utterance to a structured adjustment stored in Corrective Memory, so addressed failure modes typically need not be corrected again under the same conditions. On a real-robot desktop-clearing testbed, PhysClaw-0 matches teleoperation episode success while reducing human working time to 16%. Language corrections improve verifier-human agreement in all four evaluated settings and raise average single-attempt success from 12.5% to 47.5% (arm-selection: 20.0% to 50.0%). Policies fine-tuned on PhysClaw-0 data match teleoperation-trained policy success at a fraction of collection human cost.