AI summaryⓘ
The authors developed a method called Anatomy Contextualized Adaptation (ACA) to improve how CT scan models understand detailed body parts while still keeping the big picture. Instead of retraining big models from scratch, ACA uses existing models and breaks down scans into specific anatomy parts, then learns how these parts relate to each other and the overall scan through text descriptions. This approach works better and faster than previous methods for recognizing findings in CT scans without extra training. The authors also found that ACA learns meaningful connections between different body parts. Overall, ACA offers an efficient way to align detailed anatomy information with language while keeping the global context intact.
CT (Computed Tomography)Vision-Language ModelsFoundation ModelsFine-grained Pre-trainingAnatomy-level EmbeddingsTransformersTotalSegmentatorZero-shot ClassificationRadiology Reports
Authors
Roshan Kenia, Stephanie L McNamara, William Lotter
Abstract
CT vision-language foundation models have demonstrated promising performance across downstream tasks, but are typically trained with whole-volume representations that dilute fine-grained anatomical signals. Fine-grained vision-language pre-training addresses this by aligning anatomy-level visual features with anatomy-specific text, but in doing so discards the global context that whole-volume models provide. Furthermore, existing fine-grained approaches train from scratch, making them computationally expensive. We introduce Anatomy Contextualized Adaptation (ACA), a lightweight framework that adapts frozen CT foundation model representations for anatomy-level vision-language alignment while enhancing global contextualization. ACA uses TotalSegmentator to decompose CT volumes into anatomy-level embeddings, which are refined via a transformer that captures cross-anatomy relationships, and aligned to both per-anatomy and scan-level text extracted from radiology reports. Evaluated on Merlin and CT-RATE, ACA consistently outperforms both the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification, while requiring less than one hour of training once embeddings are cached. The attention weights learned by ACA's inter-anatomy transformer additionally indicate plausible cross-anatomy context routing. Altogether, these results support ACA as a lightweight approach for adapting CT foundation models to anatomically grounded vision-language alignment while preserving and enhancing global anatomical context.