OSCAR: Omni-Embodiment Skeleton-Conditioned World Action Model for Robotics
2026-06-03 • Robotics
Robotics
AI summaryⓘ
The authors introduce OSCAR, a video model that predicts robot actions and works across different types of robot arms and even human hands. They solved problems like limited training data and poor generalization by combining diverse robotics and human datasets and using a simple skeleton-based way to represent actions. Their model improves how well it follows actions, looks realistic, and stays consistent over time compared to other methods. They also show that OSCAR's virtual tests of robot policies match up well with how robots perform in real life.
video world modelrobot embodimentaction-conditioned predictionkinematic skeletonpolicy evaluationrobotics datasetsCosmos-Predict2.5-2Brobot policyvirtual simulationRoboArena
Authors
Zhuoyuan Wu, Jun Gao
Abstract
We present OSCAR, a precise action-conditioned video world model that generalizes across different robot embodiments and enables robot policy evaluation. Existing video world models face three main challenges for real-world robot evaluation: limited scenario diversity in current robot training datasets, imprecise action following, and poor generalization across embodiments for broad adoption. We tackle these challenges from two perspectives. At its core is a large-scale standardized data pipeline that curates, filters, and deduplicates broad robotics and egocentric human datasets, yielding a clean joint-training dataset that spans diverse tasks, scenarios, actions, and robot embodiments. To condition the video model, we adopt 2D kinematic skeleton rendering as a unified conditioning representation that generalizes across different robot arms or even human hands. We finetune the Cosmos-Predict2.5-2B model on a single GH200 GPU. Our model achieves significant improvement on action following, appearance quality, and motion consistency, compared to existing baselines, which either have a much larger model size or require more GPUs. We further deploy OSCAR to evaluate robot policies from RoboArena. Extensive experiments demonstrate the significant correlation between our virtual policy evaluation in OSCAR and real-world evaluation, paving the way for the future where robot policies can be purely evaluated in virtual generated worlds.