NoPA: Non-Parametric Online 3D Scene Graph Generation
2026-07-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors point out that older methods for generating 3D scene graphs are slow because they rely on complex point-cloud data and environment mapping. A recent faster method uses a simple 3D Gaussian to represent each object, but this loses important shape details and causes problems when combining objects. To fix this, the authors propose NoPA, which uses a more detailed, flexible object representation without slowing down the process. They also create a new way to merge object data that keeps things accurate and fast. Their experiments show that NoPA works better than other methods while still running in real time.
3D scene graphGaussian distributionnon-parametric distributionpoint cloudkernel density estimatemaximum mean discrepancyreal-time inferenceobject mergingparticle representation
Authors
Qi Xun Yeo, Seungjun Lee, Yan Li, Gim Hee Lee
Abstract
Classic 3D scene graph generation approaches fail to work in real-time due to the heavy computational cost of environment mapping and the need to generate intermediate point-cloud representations. To alleviate this issue, a recent work eschews point clouds in favor of a lightweight Gaussian distribution for each object. This approximation drastically speeds up inference and enables real-time 3D scene graph generation. However, the representation has two key weaknesses. \textbf{1)} Each object is approximated by a single 3D Gaussian, which causes a severe loss of 3D geometric detail. \textbf{2)} The discrepancy between this approximation and the true object geometry exacerbates the inaccurate merging of object candidates during online inference. To address these issues, we propose \textbf{NoPA}, which represents each object as a separate non-parametric distribution. This formulation retains 3D geometric information while preserving real-time inference of the parametric Gaussian formulation. To build upon our novel object representation, we propose a tailored merging strategy to recover coherent object instances. Specifically, we leverage maximum mean discrepancy on kernel density estimates to enable robust merging of object candidates during online exploration while minimizing added computational complexity. The key is to maintain a fixed particle set per object. Furthermore, to rectify the relation loss caused by misclassified objects, NoPA propagates relationships between objects with high affinity. Experiments show that NoPA substantially outperforms current methods without sacrificing real-time inference speed.