SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization
2026-08-13 • Computation and Language
Computation and Language
AI summaryⓘ
The authors developed a method called SAEVerbalizer that helps explain features created by sparse autoencoders (SAEs) from large language models (LLMs). Instead of guessing what these features mean by watching model behavior, their approach teaches the LLM to produce natural language explanations directly from the SAE features. Their tests showed this method works well even on new features and across different models. They also found that changing combinations of features changes the explanations in predictable ways.
Sparse autoencoderLarge language modelFeature extractionNatural language explanationFine-tuningModel interpretabilityRepresentation learningDecoder directionsAdapterIntervention experiments
Authors
Weihan Meng, Hongzhu Guo, Yi Jing, Dewen Liu, Zijun Yao, Xiaozhi Wang, Lei Hou, Juanzi Li
Abstract
Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale. We introduce SAEVerbalizer, a framework that injects SAE decoder directions into an LLM's representations and fine-tunes the LLM's downstream layers to generate natural-language explanations of the injected features. Once trained, the resulting verbalizer explains SAE features directly from decoder directions, addressing both limitations. Our experiments show that the learned verbalization capability generalizes to unseen features, transfers across separately trained SAE dictionaries, and, with a lightweight adapter, extends to SAE features from different LLMs. Intervention experiments show that injecting multiple directions yields an explanation combining their meanings, while reversing individual directions produces corresponding meaning shifts.