Automatic Laplace Collapsed Sampling: Scalable Marginalisation of Latent Parameters via Automatic Differentiation
2026-03-27 • Machine Learning
Machine Learning
AI summaryⓘ
The authors introduce Automatic Laplace Collapsed Sampling (ALCS), a method that simplifies Bayesian models by automatically reducing complex hidden variables into simpler forms using differentiation tools. They combine this with nested sampling to efficiently explore key parameters, making it easier to compute important Bayesian probabilities in high-dimensional problems without complicated manual math. Their approach runs efficiently on GPUs and can use more flexible approximations like the Student-t distribution for better accuracy. They tested ALCS on various models and provide a way to check when approximations work well or fail without needing expensive computations.
Bayesian modelslatent variablesautomatic differentiationLaplace approximationnested samplingmaximum a posteriori (MAP)Bayesian evidenceGPU parallelizationStudent-t distributioneffective sample size (ESS)
Authors
Toby Lovick, David Yallup, Will Handley
Abstract
We present Automatic Laplace Collapsed Sampling (ALCS), a general framework for marginalising latent parameters in Bayesian models using automatic differentiation, which we combine with nested sampling to explore the hyperparameter space in a robust and efficient manner. At each nested sampling likelihood evaluation, ALCS collapses the high-dimensional latent variables $z$ to a scalar contribution via maximum a posteriori (MAP) optimisation and a Laplace approximation, both computed using autodiff. This reduces the effective dimension from $d_θ+ d_z$ to just $d_θ$, making Bayesian evidence computation tractable for high-dimensional settings without hand-derived gradients or Hessians, and with minimal model-specific engineering. The MAP optimisation and Hessian evaluation are parallelised across live points on GPU-hardware, making the method practical at scale. We also show that automatic differentiation enables local approximations beyond Laplace to parametric families such as the Student-$t$, which improves evidence estimates for heavy-tailed latents. We validate ALCS on a suite of benchmarks spanning hierarchical, time-series, and discrete-likelihood models and establish where the Gaussian approximation holds. This enables a post-hoc ESS diagnostic that localises failures across hyperparameter space without expensive joint sampling.