Structure-aware divergences for comparing probability distributions
2026-03-23 • Information Theory
Information Theory
AI summaryⓘ
The authors introduce new ways to measure differences between groups that take into account how the elements are related or similar, unlike traditional methods that treat all elements as totally separate. These new measures are based on a concept called Bregman divergences and keep some nice features of well-known information theory tools. They work well in experiments, detecting patterns missed by older methods and running faster than certain complex techniques. The authors show these methods help understand real-world problems, like how jobs cluster in regions or how species diversity is distributed in nature, by considering the structure of relationships.
probability distributionsBregman divergencesKullback-Leibler divergenceShannon entropyoptimal transportclusteringeconomic geographyfunctional beta-diversityinformation theorysynthetic datasets
Authors
Rohit Sahasrabuddhe, Renaud Lambiotte
Abstract
Many natural and social science systems are described using probability distributions over elements that are related to each other: for instance, occupations with shared skills or species with similar traits. Standard information theory quantities such as entropies and $f$-divergences treat elements interchangeably and are blind to the similarity structure. We introduce a family of divergences that are sensitive to the geometry of the underlying domain. By virtue of being the Bregman divergences of structure-aware entropies, they provide a framework that retains several advantages of Kullback-Leibler divergence and Shannon entropy. Structure-aware divergences recover planted patterns in a synthetic clustering task that conventional divergences miss and are orders of magnitude faster than optimal transport distances. We demonstrate their applicability in economic geography and ecology, where structure plays an important role. Modelling different notions of occupation relatedness yields qualitatively different regionalisations of their geographic distribution. Our methods also reproduce established insights into functional $β$-diversity in ecology obtained with optimal transport methods.