Provably Reduced Sample Cost in Prior-Guided Hyperparameter Optimization
2026-06-03 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how using prior knowledge can make hyperparameter optimization (HPO) faster and less costly in automated machine learning. They provide mathematical bounds showing how good priors help reduce the number of tests needed to find the best settings. If the prior information is accurate, fewer evaluations are required, but if it's wrong, performance is similar to not using priors at all. They confirm their findings with experiments on synthetic and real deep learning benchmarks, achieving significant savings in computational effort. This work helps create more efficient and eco-friendly AutoML systems by guiding the optimization process better.
hyperparameter optimizationautomated machine learningmulti-fidelity optimizationsample complexityprior distributionsbest-arm identificationcomputational efficiencyperformance boundsLCBenchsynthetic benchmarks
Authors
Leona Hennig, Jasmin Brandt, Lukas Fehring, Barbara Hammer, Marius Lindauer, Marcel Wever
Abstract
Large-scale hyperparameter optimization (HPO) in automated machine learning (AutoML) consumes substantial computational resources, raising growing concerns about scalability and energy efficiency. Existing methods use prior information heuristically to accelerate both black-box and multi-fidelity settings, but they lack a characterization of how prior informativeness quantitatively reduces sample complexity. In this work, we provide the first distribution-dependent sample complexity bounds for multi-fidelity HPO with priors through the formal lens of fixed-budget best-arm identification. By modeling priors directly over arm means as configuration performance, we derive explicit, distribution-dependent error bounds that quantify the relationship between priors and evaluation budget. Our analysis shows that informative priors, which concentrate probability mass on near-optimal arms, yield reductions in the number of required evaluations, whereas baseline performance is recovered with uninformative or misleading priors. We conduct proof-of-concept experiments on a synthetic benchmark and on LCBench, a common multi-fidelity HPO benchmark for deep learning, to confirm our theoretical results, achieving up to 90% budget reduction while retaining solution quality. Together, our results provide a principled foundation for prior-guided and compute-efficient green AutoML.