UniSHARP: Universal Sharp Monocular View Synthesis
2026-06-05 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed UniSHARP, a new method that improves a previous photorealistic view synthesis technique called SHARP. Their goal was to make it work with many different types of cameras, including fisheye and panoramic cameras, not just regular ones. They did this by creating a special way to align images from different cameras into a common format using features and Gaussian primitives. They tested their method on a new benchmark with a variety of camera types and found that UniSHARP performed significantly better than other methods.
Photorealistic view synthesisMonocular renderingPinhole camera modelFisheye cameraOmnidirectional panoramic imagingFeature alignmentGaussian primitives3D spatial featuresRay-based representationBenchmark evaluation
Authors
Meixi Song, Dizhe Zhang, Hao Ren, Ruiyang Zhang, Bo Du, Ming-Hsuan Yang, Lu Qi
Abstract
In this work, we focus on extending SHARP, the popular photorealistic view synthesis method, for universal monocular rendering across a continuum of camera systems, from conventional perspective cameras to wide-field-of-view, fisheye and omnidirectional panoramic settings. To overcome the pinhole-specific assumptions of SHARP, our key idea is to align various images in a unified omnidirectional latent space. Thus, we propose UniSHARP, which performs implicit alignment in both feature and Gaussian spaces. Specifically, Gaussian primitives are arranged along rays and radial distances in a ray-based universal representation, while 2D semantic and 3D spatial features extracted from UniK3D-inspired encoders are jointly decoded to generate the complete Gaussian cloud. To comprehensively evaluate our method, we construct a benchmark covering diverse imaging systems across various scenes. The benchmark is further stratified by field of view (FoV) to enable fine-grained assessment of the universal monocular rendering task. Extensive experiments on the proposed benchmark demonstrate the effectiveness of UniSHARP, outperforming alternative methods by a large margin. The project page can be found at: https://insta360-research-team.github.io/Unisharp-website/