Finite-Temperature de Bruijn Identities: Fisher Information as the Spectral Gap of Blahut--Arimoto Dynamics
2026-06-02 • Information Theory
Information Theory
AI summaryⓘ
The authors extend a well-known formula connecting entropy and Fisher information, called de Bruijn's identity, to apply at any 'temperature' rather than just zero temperature. They use a mathematical framework from spectral theory related to rate-distortion optimization, showing that a key quantity called the spectral gap acts like a temperature-regularized version of Fisher information. This leads them to an exact version of de Bruijn's identity valid for all temperatures, revealing a deeper structure behind the classical result. Their work suggests new insights into how information measures relate geometrically and dynamically.
de Bruijn's identitydifferential entropyFisher informationrate-distortion optimizationBlahut–Arimoto algorithmspectral gapspectral theoryfree energyGaussian convolutionentropy power inequality
Authors
Qiao Wang
Abstract
We uncover a finite-temperature extension of de Bruijn's identity -- the classical relation $\frac{d}{dt}h(X+\sqrt{t}Z)=\frac{1}{2}J(X)$ connecting differential entropy and Fisher information. Our framework is the spectral theory of Blahut--Arimoto (BA) dynamics, recently developed by Wang~\cite{Wang2026} for the analysis of rate-distortion optimization. The central observation is elementary yet profound: for Gaussian sources, the spectral gap $\lam$ of the BA relaxation kernel $\G$ satisfies $\lam = 1/(2βσ^2)$~\cite{Wang2026}, while the Fisher information of the source is $J = 1/σ^2$. Hence \[ {\lam = \frac{J}{2β}} \] for all inverse temperatures $β> 1/(2σ^2)$. This identifies the BA spectral gap as a \emph{finite-temperature regularization of Fisher information}. From this observation we derive an exact finite-temperature de Bruijn identity: \[ \frac{\partial F_β}{\partial σ^2} = \frac{1}{2βσ^2} = \lam, \] where $F_β$ is the BA free energy. This identity holds for all finite $β$ without any limit procedure. The classical de Bruijn identity follows as the exact consequence $β\,\partial F_β/\partialσ^2 = J/2$. The significance is structural: classical de Bruijn is not an isolated fact about Gaussian convolutions, but the $β\to\infty$ shadow of a one-parameter family of exact identities living in the spectral geometry of rate-distortion optimization. We discuss implications for the entropy power inequality, the $χ^2$-dissipation structure of BA dynamics, and the geometric unification of information inequalities.