Fundamental limits of distributed multiclass classification from simple binary decisions
2026-07-21 • Information Theory
Information TheoryMachine Learning
AI summaryⓘ
The authors study how to build a system that can sort items into K categories by combining only about log(K) simple yes-or-no decisions. They focus on the case where each simple decision is based on dividing data with a flat boundary (a hyperplane). Using a simplified model where categories and data points follow Gaussian distributions, the authors calculate exact limits on how well this approach can work. They also run experiments that support their theoretical findings.
K-class classificationbinary classifiershyperplanesGaussian distributiondimensionalitydistributed classificationperformance boundsdecoding regimes
Authors
Ioannis Papageorgiou, Srinivas Nomula, Ayalvadi Ganesh, Sidharth Jaggi, Parimal Parag
Abstract
We consider the problem of constructing a $K$-class classifier from the combination of $O(\log K)$ simple binary classifiers -- this is a natural paradigm to construct a sophisticated classifier in a distributed manner with each agent performing a relatively straightforward task. We study the fundamental performance limits of such a classifier when the corresponding binary classifiers are hyperplanes. For a stylized Gaussian setting where the $K$ class centers are independent Gaussian points in $\mathbb R^d$ and the observations are corrupted by Gaussian noise, we derive explicit performance bounds across several decoding and dimensional regimes. Extensive simulation experiments provide strong empirical validation of the presented theoretical results.