AI summaryⓘ
The authors address how to reliably pick individuals for treatment based on predictions of how much they will benefit, even when these predictions have errors. They focus on controlling the rate of false positives (false discovery rate) while working with complex models that don't show their inner workings (black-box). Because actual errors are hidden, they create a special error estimate using proxy outcomes, but this can get noisy when data variability is high. To fix this, the authors introduce a method that reduces noise by adjusting for variance and combines statistical calibration with a selection procedure to keep errors low. Their approach improves the ability to correctly select individuals while still controlling mistakes, as shown in their experiments.
Selective deploymentConditional Average Treatment Effect (CATE)False Discovery Rate (FDR)Doubly robust estimationProxy errorsHeteroskedasticityConformal calibrationBenjamini--Hochberg procedureVariance estimationBlack-box predictors
Authors
Xinyun Lu, Haoang Chi, Zhiheng Zhang
Abstract
In selective deployment, practitioners act only on a model-chosen subset of individuals based on predicted conditional average treatment effects, but marginal conformal guarantees need not control reliability on that selected subset. We study reliable selection for black-box CATE predictors: selecting candidates whose CATE errors are below a tolerance while controlling the false discovery rate (FDR). Since CATE errors are unobservable, we construct doubly robust proxy errors from pseudo-outcomes; however, naive proxies can lose power under heteroskedasticity because variance overwhelms the reliability signal. We propose Denoised Conformal Alignment, which subtracts an estimated conditional variance component and combines conformal calibration with Benjamini--Hochberg selection. Our analysis shows that validity is governed by stability of proxy/oracle threshold labels, rather than pointwise perfection of the variance estimator. Experiments show substantially improved power while maintaining FDR control across challenging settings.