TSR-Ego: Temporally Guided Stereo Refinement Framework for Egocentric 3D Human Pose Estimation
2026-07-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem of estimating 3D human poses from head-mounted stereo cameras, which is hard due to distorted views and parts of the body being hidden or out of sight. They introduce TSR-Ego, a method that uses past video frames to improve the accuracy of current pose estimates by integrating temporal information earlier in the feature processing. Their approach combines motion cues with stereo vision in a way that works online without needing future frames. Tests show this method performs better than previous ones, especially in real-world situations.
3D human pose estimationegocentric visionstereo camerastemporal convolutiondeformable attentionself-occlusionfisheye distortioncross-attentiononline inferencetransformers
Authors
Md Mushfiqur Azam, John Quarles, Kevin Desai
Abstract
Egocentric 3D human pose estimation from head-mounted stereo cameras is challenging due to fisheye distortion, severe self-occlusion, and frequent truncation of body joints outside the camera field of view. Recent stereo egocentric methods have improved performance through heatmap lifting, stereo correspondence, and transformer-based refinement, but they often rely heavily on frame-local evidence or use temporal information only as auxiliary pose-level context. This limits robustness when current-frame stereo cues are weak, occluded, or ambiguous. We propose TSR-Ego, a temporally guided stereo framework that couples short-term motion evidence with projection-guided feature sampling. The model first enriches dense stereo feature maps using a causal depthwise-separable temporal convolution, allowing past visual evidence to influence the feature space before deformable cross-attention. A single-stage causal stereo decoder then refines learned 3D joint queries through temporal self-attention, joint self-attention, and fisheye deformable stereo cross-attention, using the evolving pose estimate to generate 2D sampling references. Unlike methods that apply temporal reasoning mainly after pose prediction, TSR-Ego uses motion context to shape both the sampled stereo features and the joint representations while preserving online inference without future frames. Experiments on UnrealEgo2 and UnrealEgo-RW show state-of-the-art performance, with especially strong gains on real-world sequences.