Fast 3D Foundation Model Initialized Gaussian Splatting
2026-07-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionGraphicsRobotics
AI summaryⓘ
The authors present a quick method to build detailed 3D models using a technique called 3D Gaussian Splatting without relying on the traditional Structure-from-Motion process. They use pre-trained 3D models to guess camera positions and point clouds, then improve these guesses using depth information to speed up learning. Their approach works well even with fewer pictures and includes a special module to make camera positioning more accurate in sparse views. Tests show it produces quality 3D reconstructions much faster than existing methods, making it useful for applications like robots and virtual reality.
3D Gaussian SplattingStructure-from-Motion (SfM)3D Foundation ModelsCamera Pose EstimationPoint-Cloud InitializationDepth-Guided LossMLP-Based Pose RefinementSparse-View ReconstructionPSNRLPIPS
Authors
Anurag Dalal, Daniel Hagen, Kjell G. Robbersmyr, Kristian Muri Knausgård
Abstract
This paper introduces a fast method for high-quality 3D Gaussian Splatting (3DGS) reconstruction without traditional Structure-from-Motion (SfM). The proposed approach leverages 3D Foundation Models (3DFMs) for camera pose and point-cloud initialization, then jointly optimizes both camera poses and Gaussian primitives using a depth-guided loss function. This enables fast convergence even from rough initialization with as few as 50-60 input views. To further improve reconstruction quality in sparse-view scenarios, an MLP-based pose refinement module is introduced alongside depth-guided supervision from the foundation model. Extensive experiments on Mip-NeRF 360, Tanks and Temples, and RobustNeRF demonstrate that the proposed method achieves competitive reconstruction quality (23.61 dB PSNR, 0.19 LPIPS) while reducing training time to approximately three minutes per scene. The proposed method produces ready-to-use 3DGS models at a fraction of the time required by existing pipelines, making it suitable for near real-time applications in robotics, VR, and autonomous navigation.