A Large-Scale Empirical Comparison of Meta-Learners and Causal Forests for Heterogeneous Treatment Effect Estimation in Marketing Uplift Modeling
2026-04-07 • Machine Learning
Machine Learning
AI summaryⓘ
The authors tested four methods to predict how likely different customers are to respond positively to marketing, using a huge dataset from Criteo. They found that one method, called S-Learner, worked best at identifying customers who would actually buy more when targeted. They also used a tool called SHAP to figure out which features mattered most in predicting responses and studied how certain the predictions were. Their work helps businesses choose the best techniques for big marketing campaigns based on solid evidence.
Conditional Average Treatment Effect (CATE)Uplift ModelingS-LearnerT-LearnerX-LearnerCausal ForestLightGBMQini CoefficientSHAP AnalysisPropensity Score
Authors
Aman Singh
Abstract
Estimating Conditional Average Treatment Effects (CATE) at the individual level is central to precision marketing, yet systematic benchmarking of uplift modeling methods at industrial scale remains limited. We present UpliftBench, an empirical evaluation of four CATE estimators: S-Learner, T-Learner, X-Learner (all with LightGBM base learners), and Causal Forest (EconML), applied to the Criteo Uplift v2.1 dataset comprising 13.98 million customer records. The near-random treatment assignment (propensity AUC = 0.509) provides strong internal validity for causal estimation. Evaluated via Qini coefficient and cumulative gain curves, the S-Learner achieves the highest Qini score of 0.376, with the top 20% of customers ranked by predicted CATE capturing 77.7% of all incremental conversions, a 3.9x improvement over random targeting. SHAP analysis identifies f8 as the dominant heterogeneous treatment effect (HTE) driver among the 12 anonymized covariates. Causal Forest uncertainty quantification reveals that 1.9% of customers are confident persuadables (lower 95% CI > 0) and 0.1% are confident sleeping dogs (upper 95% CI < 0). Our results provide practitioners with evidence-based guidance on method selection for large-scale uplift modeling pipelines.