NeRFscopy: Neural Radiance Fields for in-vivo Time-Varying Tissues from Endoscopy
2026-02-17 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created NeRFscopy, a new method to make 3D models from videos taken inside the body with an endoscope camera. Their method can handle moving and changing tissues, using only a single camera and no prior templates or training. It learns from the images themselves to create detailed 3D views and can show new angles that weren’t originally recorded. The authors show that NeRFscopy works better than other methods in tough endoscopy scenes.
endoscopy3D reconstructionneural renderingmonocular cameradeformable tissuesradiance fieldSE(3) transformationsnovel view synthesisself-supervised learning
Authors
Laura Salort-Benejam, Antonio Agudo
Abstract
Endoscopy is essential in medical imaging, used for diagnosis, prognosis and treatment. Developing a robust dynamic 3D reconstruction pipeline for endoscopic videos could enhance visualization, improve diagnostic accuracy, aid in treatment planning, and guide surgery procedures. However, challenges arise due to the deformable nature of the tissues, the use of monocular cameras, illumination changes, occlusions and unknown camera trajectories. Inspired by neural rendering, we introduce NeRFscopy, a self-supervised pipeline for novel view synthesis and 3D reconstruction of deformable endoscopic tissues from a monocular video. NeRFscopy includes a deformable model with a canonical radiance field and a time-dependent deformation field parameterized by SE(3) transformations. In addition, the color images are efficiently exploited by introducing sophisticated terms to learn a 3D implicit model without assuming any template or pre-trained model, solely from data. NeRFscopy achieves accurate results in terms of novel view synthesis, outperforming competing methods across various challenging endoscopy scenes.