LongStream: Long-Sequence Streaming Autoregressive Visual Geometry

2026-02-13Computer Vision and Pattern Recognition

Computer Vision and Pattern Recognition
AI summary

The authors introduce LongStream, a new method for building 3D models from very long video sequences. Unlike older methods that rely on the first frame and cause errors over time, LongStream predicts positions relative to recent keyframes, making the task easier and more stable. They also separate how the system learns size and shape to reduce errors, and improve how the model remembers past information using a special training technique. Their experiments show LongStream can reconstruct scenes accurately and smoothly over long distances and many frames.

3D reconstructionautoregressive modelskeyframe posesscale driftTransformerattention mechanismcache consistencymetric-scale reconstructionlong-sequence processingvisual geometry
Authors
Chong Cheng, Xianda Chen, Tao Xie, Wei Yin, Weiqiang Ren, Qian Zhang, Xiaoyuang Guo, Hao Wang
Abstract
Long-sequence streaming 3D reconstruction remains a significant open challenge. Existing autoregressive models often fail when processing long sequences. They typically anchor poses to the first frame, which leads to attention decay, scale drift, and extrapolation errors. We introduce LongStream, a novel gauge-decoupled streaming visual geometry model for metric-scale scene reconstruction across thousands of frames. Our approach is threefold. First, we discard the first-frame anchor and predict keyframe-relative poses. This reformulates long-range extrapolation into a constant-difficulty local task. Second, we introduce orthogonal scale learning. This method fully disentangles geometry from scale estimation to suppress drift. Finally, we solve Transformer cache issues such as attention-sink reliance and long-term KV-cache contamination. We propose cache-consistent training combined with periodic cache refresh. This approach suppresses attention degradation over ultra-long sequences and reduces the gap between training and inference. Experiments show LongStream achieves state-of-the-art performance. It delivers stable, metric-scale reconstruction over kilometer-scale sequences at 18 FPS. Project Page: https://3dagentworld.github.io/longstream/