DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees
2026-08-13 • Machine Learning
Machine Learning
AI summaryⓘ
The authors present DARTree, a new method to speed up language models that generate text step-by-step. Unlike previous approaches that check one possible text draft at a time, DARTree creates and evaluates many drafts in a tree-like structure all at once, making the process more efficient. It uses a pretrained correction tool to improve these drafts and then picks the best ones quickly without slowing down with repeated sorting steps. Their tests show that DARTree accepts more tokens per check and runs faster than earlier methods on various tasks like math and coding. This results in significant speed improvements without losing the quality of generated text.
autoregressive language modelspeculative decodingdiffusion-based drafterstoken block predictioncorrection headcandidate treebest-first pruningacceptance lengthspeedupverification round
Authors
Tianyi Li, Yaxin Luo, Xinyi Shang, Zhiqiang Shen
Abstract
Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block in parallel, but their position-wise distributions are marginal rather than conditioned on tokens selected along each draft path. Existing recurrent correction incorporates causal information along a single draft chain, whereas diffusion-based tree construction broadens candidate coverage without carrying this correction along individual branches. We introduce DARTree, a training-free speculative decoding method that extends a pretrained AR correction head from chains to trees. DARTree first constructs a fixed-width candidate tree by expanding and scoring all nodes at each depth in a single batch, and then only applies best-first pruning to select the verification tree, decoupling AR-head inference from sequential heap operations. Across seven math, code, and chat benchmarks, DARTree achieves the highest average acceptance length and speedup in all four model--temperature configurations, accepting up to 12.97 tokens per verification round, 98.6\% more than DFlash and 27.9\% more than Domino in the same setting, and reaching up to 9.73$\times$ lossless speedup over locally measured autoregressive decoding.