PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives
2026-08-13 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors focus on video world models, which predict future video frames based on current views and user actions. They point out that comparing these models is hard because humans might use different actions to achieve the same goal in each model. To fix this, the authors use AI 'Agent Players' to test how well these models reach long-term goals in 171 different scenarios, checking things like if the scene stays consistent and reacts realistically. Their tests show that current models struggle with maintaining spatial consistency and evolving states over time when faced with long interactions.
Video world modelsLong-horizon objectivesAgent PlayersSpatial consistencyInteraction fidelityOut-of-sight evolutionInsight evolutionBenchmarkControllabilityVideo quality
Authors
Kaixin Ding, Xi Chen, Minghong Cai, Zhiyuan Xu, Yiyang Wang, Yuxiang Lu, Junyi Li, Shuyang Chen, Yuan Gao, Xin Tao, Pengfei Wan, Hengshuang Zhao
Abstract
Video world models simulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consistency and action controllability over long sequences. However, fairly comparing these interactive models remains challenging. In practice, a human player typically evaluates a world model by pursuing long-horizon objectives through interaction. For example, a user may turn around 360 degrees to see whether the environment remains consistent, or walk into the water and inspect whether realistic water ripples are generated. The action sequence required to achieve the same objective may vary substantially between models, making fixed action-conditioned evaluation unsuitable for cross-model comparison. To address this, we employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives. Building on this paradigm, we introduce PlayWorld, a benchmark providing 171 scenarios, each with a specified objective. To evaluate performance thoroughly, we assess models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution. In addition, we incorporate basic ability metrics for video quality and controllability. Experiments across nine state-of-the-art world models reveal that current models remain unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution. Code and data are available at https://github.com/kxding/PlayWorld.