Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction
2026-05-29 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors present C4G, a new method for reconstructing moving 3D scenes from single-camera videos. Instead of predicting lots of separate 3D pieces for each frame, C4G uses a small set of special tokens that capture motion over time, making the 3D scene more consistent and easier to learn. They also add a video enhancement step to get finer details. Their method works well even with big time gaps and doesn't need camera positions, making it simpler and more robust for understanding and tracking motion in videos.
monocular video3D reconstruction3D Gaussianstemporal contextnovel-view synthesisvideo diffusion modelfeature aggregationpoint trackingdynamic scene understandingcamera pose
Authors
Mungyeom Kim, Minkyeong Jeon, Honggyu An, Jaewoo Jung, Hyuna Ko, Jisang Han, Hyeonseo Yu, Donghwan Shin, Sunghwan Hong, Takuya Narihira, Kazumi Fukuda, Yuki Mitsufuji, Seungryong Kim
Abstract
Dynamic scene reconstruction from monocular video remains a fundamental challenge in computer vision. Existing feed-forward methods predict 3D Gaussians pixel-wise for each frame, suffering from duplicated Gaussians and view-dependent biases that hinder effective learning of scene motion. We present C4G, a feed-forward 4D reconstruction framework built upon a compact set of timestamp-conditioned learnable Gaussian query tokens. Each token aggregates corresponding features across the full temporal context and decodes a 3D Gaussian whose position is modulated by the target timestamp, enabling globally coherent motion modeling without per-scene optimization. To capture fine-grained details, we further introduce a video diffusion model-based rendering enhancement module. Since our framework effectively aggregates features into Gaussians, we extend this capability to feature lifting, producing a 4D feature field that supports point tracking and dynamic scene understanding. C4G achieves strong novel-view synthesis performance using significantly fewer Gaussians and without requiring camera poses, while exhibiting stronger motion modeling and robustness to large temporal gaps.