PhysiFormer: Learning to Simulate Mechanics in World Space
2026-06-25 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors introduce PhysiFormer, a new model that predicts how 3D objects move in a way that looks physically realistic. Instead of working with 2D videos, their model uses the actual 3D shape of objects and predicts how each point on the shape moves over time. They do this by treating the prediction as a special kind of noise-removal process, which lets the model guess different possible futures for the object’s motion. Their approach works well on objects that are either hard or soft, and it can handle multiple objects without needing them to be labeled separately. This makes it useful for things like robotics or animation where understanding real 3D movement is important.
3D meshdiffusion processvertex trajectoryrigid body dynamicselasticitydenoising diffusion modelattention mechanismpermutation invarianceworld coordinatesphysical consistency
Authors
Yiming Chen, Yushi Lan, Andrea Vedaldi
Abstract
We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, as well as object material type, rigid or elastic, the model samples future vertex trajectories. While related neural physics approaches build on ad-hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer shows that excellent results can be obtained without any such inductive biases, by casting vertex trajectory prediction as a single denoising diffusion process directly in world coordinates. The probabilistic formulation captures uncertainty in the learned dynamics, enabling diverse plausible futures from initial conditions, making this framework potentially useful for applications with unobserved uncertainty. The model features attention factorised over time, space, and objects for efficiency, enabling permutation-invariant multi-object reasoning without needing explicit object encoding. Trained on over 100k simulated trajectories, PhysiFormer generates rigid and elastic mechanics, and generalises to mixed-material settings, unseen real-world geometries, and larger object counts. It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency. Our results position coordinate-space diffusion as a promising step toward view-invariant, geometry-aware world modelling for robotics, graphics, and physical design. Visualisations, code, and models are available at https://yimingc9.github.io/physiformer.