Conditional Diffusion Sampling
2026-05-05 • Machine Learning
Machine Learning
AI summaryⓘ
The authors address the problem of sampling from complicated probability distributions that are hard to evaluate. They combine two existing methods: Parallel Tempering (PT), which is good at exploring broadly, and diffusion-based sampling, which smoothly transforms samples but usually needs neural networks. Their new method, Conditional Diffusion Sampling (CDS), uses PT first to get a starting set of samples and then applies a special mathematical process that doesn't require neural networks to refine those samples. Their experiments show that this combination can balance sample accuracy and computation cost better than current methods.
SamplingMultimodal distributionsParallel TemperingDiffusion processesStochastic differential equationsUnnormalized densitiesMarkov Chain Monte CarloNeural approximationTransport dynamicsDensity evaluation
Authors
Francisco M. Castro-Macías, Pablo Morales-Álvarez, Saifuddin Syed, Daniel Hernández-Lobato, Rafael Molina, José Miguel Hernández-Lobato
Abstract
Sampling from unnormalized multimodal distributions with limited density evaluations remains a fundamental challenge in machine learning and natural sciences. Successful approaches construct a bridge between a tractable reference and the target distribution. Parallel Tempering (PT) serves as the gold standard, while recent diffusion-based approaches offer a continuous alternative at the cost of neural training. In this work, we introduce Conditional Diffusion Sampling (CDS), a framework that combines these two paradigms. To this end, we derive Conditional Interpolants, a class of stochastic processes whose transport dynamics are governed by an exact, closed-form stochastic differential equation (SDE), requiring no neural approximation. Although these dynamics require sampling from a non-trivial initialization distribution, we show both theoretically and empirically that the cost of this initialization diminishes for sufficiently short diffusion times. CDS leverages this by a two-stage procedure: (1) PT is used to efficiently sample the initial distribution, and then (2) samples are transported via the transport SDE. This combination couples the robust global exploration of PT with efficient local transport. Experiments suggest that CDS has the potential to achieve a superior trade-off between sample quality and density evaluation cost compared to state-of-the-art samplers.