Seg2Track++: Probabilistic Track Validation and Data Association for Multi-Object Tracking and Segmentation
2026-06-02 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed Seg2Track++, a system that helps robots and autonomous vehicles keep track of multiple moving objects more reliably. They combined a powerful segmentation model called SAM2 with a new tracking method that matches objects over time more accurately. Their approach uses smart ways to link object masks across video frames and filters out false detections to avoid confusion. Tests on a standard benchmark showed the system keeps object identities better and reduces errors without needing extra training.
Multi-Object Tracking and Segmentation (MOTS)Foundation ModelsSAM2Instance SegmentationTrack AssociationMask Centroid Distance (MCD)Confidence-Aware Cost Modulation (CCM)Probabilistic Track Validation (PTV)Bernoulli FilterKITTI MOTS Dataset
Authors
Diogo Mendonça, Tiago Barros, Cristiano Premebida, Urbano J. Nunes
Abstract
Autonomous systems require robust Multi-Object Tracking and Segmentation (MOTS) to operate reliably in dynamic environments, ensuring consistent object identities and precise mask-level delineation. Foundation models such as SAM2 have shown strong zero-shot generalization for segmentation, but their direct application to MOTS is limited by unreliable track association and false-positive propagation. This work introduces Seg2Track++, a framework that integrates instance segmentation with SAM2 and a novel track management module to perform zero-shot MOTS with enhanced temporal consistency. Tracks are associated using Mask Centroid Distance (MCD) and Confidence-Aware Cost Modulation (CCM), while Probabilistic Track Validation (PTV) employs a Bernoulli filter to validate track existence and suppress ghost tracks. Experimental results on KITTI MOTS demonstrate improved identity preservation, reduced false-positive propagation, and robust track management without fine-tuning.