A Geometric Theory of Robust Fairness Audits
2026-08-25 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how fair an algorithm seems when checking similar people’s predictions, called neighborhood-based fairness audits. They find that small changes in data can change who counts as 'neighbors,' making fairness checks unstable even if the algorithm's predictions don’t change. They create a geometric method to understand when these neighbor groups stay the same and how changes affect fairness results. Their experiments show the method helps explain why fairness audits can be sensitive or stable.
Fairness auditIndividual fairnessNearest neighborsFeature spacePerturbationsGeometric frameworkAudit stabilityNeighborhood invarianceAudit volatilityMachine learning fairness
Authors
Binita Maity
Abstract
Neighborhood-based fairness audits evaluate individual fairness by comparing predictions among similar individuals in feature space. Despite their widespread use, little is known about the robustness of the auditing procedure itself. Because these audits rely on nearest neighbor relationships, small perturbations in feature space can alter local neighborhoods and produce different fairness assessments even when model predictions remain unchanged. We develop a geometric framework for analyzing the robustness of neighborhood-based fairness audits under bounded perturbations. Our analysis establishes sufficient conditions for neighborhood invariance, quantifies how neighborhood replacement propagates to audit instability, and introduces audit volatility, a measure of the expected sensitivity of fairness audits under repeated perturbations. Experiments on benchmark datasets support the theoretical analysis and show that the proposed framework explains the observed stability of neighborhood-based fairness audits.