PhiZero: A World Model Built Around Physical Language
2026-07-30 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors present PhiZero, a model that understands and predicts how the physical world changes by using a simplified 'physical language' instead of just trying to guess future video pixels. Unlike other models that directly predict video frames, PhiZero first describes future changes with this compact physical language and then creates the video from that description. This approach helps the model better grasp and simulate physical events. Their experiments show that PhiZero can generate consistent physical scenarios and can be used for simulations and transferring motion without prior examples.
physical world modelphysical languageself-supervisionvideo predictionworld-state transitionsreason-then-rendermotion transferaction-conditioned simulation
Authors
Shuyao Shang, Yuqi Wang, Ruopeng Gao, Xu Chen, Tieniu Tan, Lue Fan, Zhaoxiang Zhang
Abstract
We introduce PhiZero, a physical world model built around physical language, a compact discrete representation of world-state transitions. Existing physical world models typically predict future videos directly in pixel space, leaving the underlying world dynamics implicit within high-dimensional visual predictors. Motivated by humans' ability to abstract predictive structure from visual experience and organize it in natural language for explicit reasoning, we learn physical language from in-the-wild videos through self-supervision and use it to explicitly reason about how the physical world evolves. Accordingly, PhiZero adopts a reason-then-render paradigm: it first infers future world evolution as a physical-language sequence and then renders the inferred transitions into videos. Extensive experiments across generation and understanding benchmarks validate the ability of PhiZero to model physically coherent world evolution. We further show its potential for realistic and interactive world modeling, fine-grained action-conditioned simulation, and zero-shot motion transfer.