GeoTrace: Geometry-Aware Trajectory Token Compression for Video Large Language Models
2026-07-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors propose GeoTrace, a method to make video language models faster by reducing the number of visual tokens they process. Instead of compressing tokens based on simple saliency or merging, GeoTrace breaks down video data into precise skeleton tokens and event tokens that can be tracked over time without losing important details. Their approach keeps important visual information while greatly cutting computation, achieving almost the same accuracy with only 10% of the tokens and much less processing. They tested GeoTrace on multiple models and benchmarks, showing it works well in different situations.
Video Large Language Modelsvisual tokenstoken compressionspatiotemporalsaliencyskeleton tokensresidual tokenstrajectoryTFLOPsmodel efficiency
Authors
Guohuan Xie, Mengqi Lei, Chuan Shi, Wei Bao, Yue Gao, Siqi Li
Abstract
Although Video Large Language Models (Video LLMs) have shown strong performance in video understanding, their efficiency is still limited by the large number of visual tokens. Existing video token compression methods typically rely on frame-wise saliency or heuristic token merging, which can over-focus on locally salient regions and produce ambiguous fused features. To address these issues, we propose GeoTrace, a training-free spatiotemporal token compression framework that decomposes video evidence into exact skeleton tokens and traceable residual event tokens. Specifically, Contextual Farthest-Point Anchoring (CFPA) preserves salient, context-consistent, and high-coverage skeleton tokens, while Trajectory-Constrained Residual Condensation (TCRC) compresses residual tokens through one-to-one temporal trajectories and constrained near-manifold condensation, producing traceable event tokens with reduced ambiguity. We evaluate GeoTrace on four Video LLMs across four video understanding benchmarks, and the results demonstrate its effectiveness and generalization across different model architectures and scenarios. On LLaVA-OneVision, with only 10\% visual tokens retained, GeoTrace achieves a \(12.99\times\) TFLOPs reduction while preserving 99.1\% of the vanilla performance. Overall, GeoTrace offers a compact and traceable token representation for efficient and robust Video LLM inference. Code is available at \href{https://github.com/guohuan-xie/GeoTrace.git}{\texttt{Code}}.