Conformal Risk Control for Non-Monotonic Losses
2026-02-23 • Machine Learning
Machine Learning
AI summaryⓘ
The authors work on a method called conformal risk control, which helps make better predictions by managing different kinds of risks, not just simple errors. They improve the method to handle more complex scenarios where the way losses behave can be irregular and have multiple factors. Their results show that the quality of these guarantees depends on how stable the prediction algorithms are. They demonstrate their approach on tasks like image classification, tumor detection, and making fair predictions about recidivism considering multiple social groups.
conformal predictionrisk controlloss functionsalgorithm stabilityselective classificationfalse discovery rateintersection over uniontumor segmentationempirical risk minimizationdebiasing
Authors
Anastasios N. Angelopoulos
Abstract
Conformal risk control is an extension of conformal prediction for controlling risk functions beyond miscoverage. The original algorithm controls the expected value of a loss that is monotonic in a one-dimensional parameter. Here, we present risk control guarantees for generic algorithms applied to possibly non-monotonic losses with multidimensional parameters. The guarantees depend on the stability of the algorithm -- unstable algorithms have looser guarantees. We give applications of this technique to selective image classification, FDR and IOU control of tumor segmentations, and multigroup debiasing of recidivism predictions across overlapping race and sex groups using empirical risk minimization.