Activation-Based Active Learning for In-Context Learning: Challenges and Insights
2026-06-03 • Computation and Language
Computation and LanguageMachine Learning
AI summaryⓘ
The authors tested whether looking at certain parts of large language models' internal signals (called MLP activations) could help pick better examples for learning. They ran many tests with different models and tasks but found that these signals do not relate well to how good an example is. Their results showed low correlation between these signals and task performance, meaning this method isn't useful for selecting examples. They suggest that the way models pack information (superposition) might explain this and propose exploring other methods like Sparse Autoencoders in the future.
Deep active learningIn-context learningMLP activationsTransformer modelsAttention maskingSpearman correlationSuperpositionSparse AutoencodersLlama modelQwen model
Authors
Yaseen M. Osman, Geoff V. Merrett, Stuart E. Middleton
Abstract
Deep active learning has previously been explored for LLM in-context sample selection, but not with methods that utilise recent advances in understanding of transformer activations. In this paper, we test the hypothesis that model activations could provide a fine-grained signal to optimise the selection of in-context examples. We present the most comprehensive analysis to date of MLP activation-based deep active learning methods applied to in-context learning, including how different attention masking strategies impact active learning across diverse classification and generative datasets, using both Llama-3.2-3B and Qwen2.5-3B base models. However, we find a negative result: MLP outputs, viewed through the lenses of massive activations or the first four moments, do not correlate with example quality or task performance. Specifically, the absolute Spearman correlation coefficient is at most 0.33 for all tasks and models we tested, showing that such activation-based sampling should not be used for in-context learning. We hypothesise that this may be due to superposition, whereby models represent more features than they have dimensionality, suggesting that methods like Sparse Autoencoders (SAEs) may be a promising future direction.