AmbientEye: A Dataset for Pupil Segmentation under Natural Ambient Infrared Illumination
2026-06-02 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors studied if passive infrared (IR) cameras without extra IR light can detect pupils outdoors using sunlight alone. They created a large dataset, AmbientEye, with millions of outdoor eye images from diverse participants under natural sunlight. They tested a top pupil detection method on this dataset and found it works worse than on traditional datasets with active IR lighting. This shows detecting pupils outdoors with just sunlight is much harder, and AmbientEye offers a new benchmark for this real-world challenge.
eye trackingpupil detectioninfrared (IR) imagingpassive IR camerasambient lightsmart glassesdatasetimage segmentationcomputer vision
Authors
Mingyu Han, Hyunyoung Han, Nitheekulawatn Thommakoon, Gangtae Park, Jieun Han, Xucong Zhang, Ian Oakley
Abstract
Eye tracking is essential for smart glasses, as it provides insight into user attention for ambient intelligence applications. However, most existing eye-tracking systems rely on active infrared (IR) illumination, creating practical barriers to all-day outdoor use due to power consumption. In this paper, we investigate whether passive IR cameras alone, without any active IR light source, can enable reliable pupil detection in unconstrained outdoor environments, where ambient sunlight serves as the sole illumination source. To support this investigation, we introduce AmbientEye, a large-scale dataset of 2,606,225 eye images collected from 35 participants from 19 countries. It is captured outdoors under natural sunlight with two off-axis camera configurations and two sun-orientation conditions. We provide high-quality pupil annotation through SAM2 automatic segmentation, followed by refinement by human annotators. We benchmark a state-of-the-art pupil segmentation algorithm on our dataset and compare its performance with that on existing datasets under controlled IR illumination. Results reveal a substantial drop in pupil segmentation performance from 0.928 on controlled IR datasets to 0.767 on AmbientEye. This performance gap highlights the challenge of the ambient-light setting. This positions AmbientEye as a first benchmark for an unexplored and highly practical eye-tracking scenario.