Where Do We (Not) Need Temporal Context in Low-Resource Video Task Adaptation?
2026-06-02 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors studied how to efficiently adapt large video models using only a few trainable parts to save on data and computing power. They compared different ways to handle the timing information in videos within different parts of the model. Their experiments showed that how and where the model processes time-related info is very important for good video understanding. This helps identify better strategies for making video models learn from limited data.
parameter-efficient fine-tuningprobingfoundation modelsvideo understandingtemporal reasoningmodel adaptationappearance featuresmotion featuresspatially dense featureslimited data scenarios
Authors
Luc P. J. Sträter, Hazel Doughty
Abstract
Parameter-efficient fine-tuning (PEFT) and probing enable adaptation of foundation models using only a small number of trainable parameters, making it attractive for video understanding where annotation and computation are expensive. However, video PEFT has focused on adapting image-pretrained models, while standard PEFT methods can also be applied to video representations. These settings are rarely compared and both confine temporal reasoning to a single component of the model, leaving open how temporal context should be distributed across backbone, PEFT and probe. In this work we provide a systematic study of model adaptation strategies for video understanding. We evaluate methods across appearance-focused, motion-focused and spatially dense settings, with a particular focus on scenarios with limited data where parameter-efficiency is most beneficial. Our results provide new insights into PEFT and probing across settings and demonstrate the importance of temporal context allocation for effective video adaptation