Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
2026-07-11 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors tested 15 different deep learning models to see how well they identify breast cancer features in mammograms, especially when used on data that's different from what they were trained on. They found that models specifically designed for mammography worked best on average, but just training on mammography images didn’t always help with generalizing to new data. Even the best models showed varying results depending on the dataset used. Their analysis suggests it's important to evaluate these models across many different datasets to truly know how reliable their features are.
foundation modelsmammographydomain shiftout-of-distributionvision-language modelsBI-RADSlinear probefeature extractionbreast densitypretraining
Authors
Giang Nguyen, Raghav Mehta, Emma A. M. Stanley, Tian Xia, Thi Hao Nguyen, Hieu Pham, Ben Glocker
Abstract
Foundation models are increasingly used as image feature extractors for mammography, but their robustness under external domain shift remains unclear. We benchmark 15 foundation-model backbones across breast density, BI-RADS severity, and cancer status using a unified frozen-backbone linear-probe protocol, training on 3 source datasets and evaluating on 12 task-compatible out-of-distribution (OOD) datasets after label harmonization. Mammography-specific vision-language models (Mammo-FM and MaMA) provide the strongest mean OOD performance, but robustness is not explained by mammography exposure alone. DINOv3 remains a competitive vision-only baseline, and mammography-adapted pretraining does not consistently improve generalization. Dataset-level analysis further shows that even leading models show heterogeneous performance across datasets. Feature-space inspection reveals that useful representations can preserve clinical signal while retaining dataset and acquisition structure. These findings highlight dataset-level OOD evaluation as a central criterion for assessing mammography representations. Our code is publicly available: https://github.com/biomedia-mira/mammo-ood.