TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models
2026-07-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors found that medical vision-language models, which can understand images and text, perform worse when shown unfamiliar medical data. To fix this, they created TCLA, a simple method that improves model predictions using only a few example cases and without any extra training. By adjusting model outputs based on these examples, TCLA helps reduce confusion between classes and adapts to new data types. Tests on various medical images showed TCLA consistently helped the models perform better than existing methods that require extra training.
Vision-Language ModelsZero-shot LearningFew-shot AdaptationOut-of-distribution DataDomain ShiftMedical ImagingLogit CorrectionModel AgnosticInter-class DeconfusionPretrained Models
Authors
Tianyou Jiang, Ziyu Zhou
Abstract
Medical Vision-Language Models (VLMs) exhibit strong zero-shot performance, yet their effectiveness still declines on out-of-distribution (OOD) data due to domain shifts and class bias inherited from large-scale pretraining. Existing few-shot adaptation methods typically introduce additional trainable components, which can be unstable in extremely low-data regimes (e.g., 1-shot), and lack robustness on different medical data. We present TCLA, a purely training-free few-shot adaptation method for Medical VLMs, which is fast and model-agnostic. TCLA corrects inference logits based on a small set of support samples, boosting pretrained VLMs performance by improving inter-class deconfusion and reducing domain shift. Extensive experiments on nine datasets across multiple medical imaging modalities including X-ray, Ultrasound, MRI, CT, Histopathology, demonstrate that TCLA consistently improves OOD performance of Medical VLMs and, in most of cases, outperforms existing training-based adaptation methods.