Are Common Substructures Transferable? Riemannian Graph Foundation Model with Neural Vector Bundles
2026-06-02 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors study how parts of graphs called substructures can be used across different graph tasks, which was not well understood before. They propose looking at these substructures through the shapes and geometry of how graph data is represented, rather than just the graph pieces themselves. Using ideas from Riemannian geometry, they create a framework called Neural Vector Bundle that helps understand these geometric properties. They build a new method called GAUGE that learns these structures and shows better performance on tasks like predicting links without extra training. Their work provides a new way to think about transferring knowledge between graphs based on geometry.
Foundation modelsGraph representationTransferabilitySubstructuresRiemannian geometryNeural Vector BundleGAUGEDirichlet lossZero-shot link predictionGraph isomorphism
Authors
Li Sun, Zhenhao Huang, Yiding Wang, Qin Chen, Pietro Lio, Philip S. Yu
Abstract
Foundation models have sparked a revolution via a pretraining-adaptation paradigm, with recent efforts extending this success to graphs. Unlike other modalities, graphs contain rich structural patterns, yet their structural transferability remains poorly understood. Prior studies consider common substructures in the discrete realm, and we are motivated by a fundamental question: Are common substructures transferable? The underlying theory is largely underexplored. In this work, we shift toward learning transferable structures through the lens of functional behavior. Theoretically, we connect transferable substructures to intrinsic geometry of the representation space. However, characterizing such intrinsic geometry has rarely been touched. Grounded in Riemannian geometry, we develop a graph intrinsic geometry learning framework called Neural Vector Bundle, which enables parsing intrinsic geometry with local coordinates. Building on this, we design GAUGE, a pretrainable neural architecture that constructs the vector bundle, flattening geometrically compatible local coordinates, and a new Dirichlet loss, which also measures the transfer effort. We empirically validate its superior expressiveness in challenging tasks including zero-shot link prediction and graph isomorphism.