PersistGS: Differentiable Physics for Object Permanence in 4D Gaussian Splatting
2026-06-02 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionGraphics
AI summaryⓘ
The authors address a problem where parts of moving objects are hidden from all cameras, causing 3D reconstructions to lose track of those objects. They propose PersistGS, which uses physics simulations of rigid objects to predict where hidden objects go during occlusion, instead of just guessing their motion. This method better follows real-world physics like bouncing and friction, leading to more accurate tracking than previous methods. Their approach also introduces a special loss to focus on object position, improving trajectory accuracy. Tests on synthetic data show their method beats simple velocity guesses and nearly matches ideal tracking.
3D Gaussian SplattingOcclusionRigid Body SimulationPhotometric SupervisionDifferentiable SimulationSE(3) TrajectoryCollision MeshesSilhouette LossPSNRTrajectory Estimation
Authors
Adrian Ramlal, John S. Zelek
Abstract
Dynamic 3D Gaussian Splatting (3DGS) methods reconstruct time-varying scenes from synchronized multi-camera video using photometric supervision. When a moving object becomes fully occluded from all training cameras, this supervision vanishes: the Gaussians representing it receive no gradient signal and degrade. Existing approaches to incomplete observations in neural reconstruction rely on learned generative priors that prioritize visual plausibility over physical correctness. We propose $\textbf{PersistGS}$, a method that restores object permanence during occlusion by coupling differentiable rigid body simulation with 3D Gaussian Splatting. Our approach decomposes the scene into per-object Gaussians and collision meshes, estimates friction and velocity from the observed pre-occlusion trajectory via differentiable simulation, and uses the resulting SE(3) trajectory to position object Gaussians throughout the occlusion period. Because the predicted trajectory satisfies the governing equations of rigid body dynamics, it faithfully captures contact events (bounces, friction-based deceleration, direction changes) that kinematic extrapolation cannot model. We introduce a centroid silhouette loss that isolates positional gradients from appearance noise, yielding 40% lower trajectory error than photometric supervision. We evaluate using cameras withheld from training that observe the object during its occlusion. Experiments on synthetic scenes show that PersistGS outperforms constant velocity extrapolation by +2.46dB PSNR and comes within 0.19dB of a ground-truth trajectory upper bound.