Real-World Point Tracking with Verifier-Guided Pseudo-Labeling
2026-03-12 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors worked on improving long-term point tracking in real-world videos, where models trained on synthetic data often perform poorly. They created a system called verifier, which checks and picks the most reliable tracking results from different models to create better training examples without needing manual labels. Using these improved training examples, their method fine-tunes tracking models more efficiently on real videos. Tests on multiple datasets showed that their approach outperforms previous methods while using less data.
long-term point trackingsynthetic datasetsself-trainingpseudo-labelsmeta-modelfine-tuningunlabeled videostrajectory evaluationreal-world benchmarks
Authors
Görkay Aydemir, Fatma Güney, Weidi Xie
Abstract
Models for long-term point tracking are typically trained on large synthetic datasets. The performance of these models degrades in real-world videos due to different characteristics and the absence of dense ground-truth annotations. Self-training on unlabeled videos has been explored as a practical solution, but the quality of pseudo-labels strongly depends on the reliability of teacher models, which vary across frames and scenes. In this paper, we address the problem of real-world fine-tuning and introduce verifier, a meta-model that learns to assess the reliability of tracker predictions and guide pseudo-label generation. Given candidate trajectories from multiple pretrained trackers, the verifier evaluates them per frame and selects the most trustworthy predictions, resulting in high-quality pseudo-label trajectories. When applied for fine-tuning, verifier-guided pseudo-labeling substantially improves the quality of supervision and enables data-efficient adaptation to unlabeled videos. Extensive experiments on four real-world benchmarks demonstrate that our approach achieves state-of-the-art results while requiring less data than prior self-training methods. Project page: https://kuis-ai.github.io/track_on_r