From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation
2026-03-16 • Robotics
RoboticsArtificial IntelligenceComputation and LanguageComputer Vision and Pattern Recognition
AI summaryⓘ
The authors present PRIMO R1, a new AI system that watches videos of robots and actively checks how well they are doing at their tasks, rather than just describing what is happening. They use a special training method called outcome-based reinforcement learning to help the system think step-by-step about progress. Their approach also organizes video inputs by showing the start and current images to better understand the task. Experiments show PRIMO R1 is more accurate than larger models and works well even on new, unseen tasks.
video MLLMsrobotic manipulationsupervised fine-tuningreinforcement learningchain-of-thoughtprogress estimationzero-shot generalizationRoboFail benchmarkmean absolute error
Authors
Yibin Liu, Yaxing Lyu, Daqi Gao, Zhixuan Liang, Weiliang Tang, Shilong Mu, Xiaokang Yang, Yao Mu
Abstract
Accurate process supervision remains a critical challenge for long-horizon robotic manipulation. A primary bottleneck is that current video MLLMs, trained primarily under a Supervised Fine-Tuning (SFT) paradigm, function as passive "Observers" that recognize ongoing events rather than evaluating the current state relative to the final task goal. In this paper, we introduce PRIMO R1 (Process Reasoning Induced Monitoring), a 7B framework that transforms video MLLMs into active "Critics". We leverage outcome-based Reinforcement Learning to incentivize explicit Chain-of-Thought generation for progress estimation. Furthermore, our architecture constructs a structured temporal input by explicitly anchoring the video sequence between initial and current state images. Supported by the proposed PRIMO Dataset and Benchmark, extensive experiments across diverse in-domain environments and out-of-domain real-world humanoid scenarios demonstrate that PRIMO R1 achieves state-of-the-art performance. Quantitatively, our 7B model achieves a 50% reduction in the mean absolute error of specialized reasoning baselines, demonstrating significant relative accuracy improvements over 72B-scale general MLLMs. Furthermore, PRIMO R1 exhibits strong zero-shot generalization on difficult failure detection tasks. We establish state-of-the-art performance on RoboFail benchmark with 67.0% accuracy, surpassing closed-source models like OpenAI o1 by 6.0%.