Image-Domain Tilt Constrained Distributed Fusion for Maneuvering UAV Tracking with Multi-Camera Electro-Optical Observations
2026-07-01 • Robotics
Robotics
AI summaryⓘ
The authors developed a method to better predict the short-term movements of drones by using the tilt (roll and pitch) visible in their images as clues about how they are accelerating. They trained a detector to recognize these tilts from video and combined data from multiple cameras and sensors to improve tracking accuracy. Their approach helps reduce large errors when the drone maneuvers quickly or changes direction unexpectedly. Tests in simulations and real experiments showed significant improvements in prediction accuracy using these image-based tilt measurements.
UAV trackingshort-horizon predictionroll and pitchimage-domain tiltdistributed sensor fusionYOLO-OBB detectoracceleration constraintsMahalanobis gategimbal IMUextrinsic drift
Authors
Minxing Sun, Yao Mao
Abstract
Short-horizon prediction is essential for electro-optical UAV tracking, especially when the target is small, maneuvering, or intermittently observed. Image center, line-of-sight, and range measurements provide direct constraints on target position, but their constraints on acceleration are weak. As a result, prediction can lag during aggressive maneuvers. This paper proposes an image-domain tilt constrained distributed fusion method for maneuvering UAV tracking. The method uses the apparent roll and pitch of a rotorcraft target in the image as low-level maneuver cues. A weak-prior auto-labeling pipeline first generates oriented bounding box and image-domain tilt labels from synchronized video, gimbal IMU, and UAV IMU data. A YOLO-OBB detector is then trained to provide online target position and tilt measurements. The front-end Python implementation is publicly available at github.com/ShineMinxing/PythonYOLO. In the fusion stage, the UAV state is modeled by position, velocity, and acceleration. Image-domain roll and pitch are introduced as acceleration-related pseudo-observations. For distributed tracking, one mobile gimbal camera and two fixed ground cameras are fused asynchronously. Camera attitude error states are augmented into the filter to absorb extrinsic drift and cross-camera systematic inconsistency. A Mahalanobis gate with time-since-last-valid covariance widening is used to reject false detections and handle dropouts. In simulation, adding roll/pitch observations reduces the prediction RMSE from 1.991 m to 0.821 m and decreases the cumulative prediction error by 60.75\%. In real distributed experiments, a self-consistency evaluation shows an 18.10\% reduction in cumulative prediction error. The results show that image-domain tilt can provide useful acceleration constraints for robust short-horizon UAV prediction.