Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
2026-07-13 • Machine Learning
Machine LearningArtificial IntelligenceComputation and Language
AI summaryⓘ
The authors studied how large language models used as judges show bias not just in their final outputs but also in their internal thought process. They found that biased judgments correspond to specific patterns in the model’s hidden layers, which can be shifted to increase or decrease bias. This internal bias pattern can predict when a judge will make errors on new tests better than just looking at the input text. Their work connects bias to the model’s inner structure rather than treating it as random output changes.
large language modelsbiashidden statesactivation manifoldcausal controllinear projectionjudge scoringmodel interpretabilitypromptingactivation geometry
Authors
Zixiang Xu, Sixian Li, Huaxing Liu, Xiang Wang, Shuai Li, Zirui Song, Xiuying Chen
Abstract
Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge's hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts reproducing biased scoring on clean inputs and reverse shifts restoring baseline scoring on biased ones, while matched-norm random directions produce shifts an order of magnitude smaller. Operational: a simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks, substantially outperforming text-based alternatives. Reading bias as activation geometry, rather than as input-output noise, unifies geometric structure, causal control, and operational prediction within a single framework. The project page is available at https://xzx34.github.io/unfair-judge/