TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding
2026-07-03 • Computation and Language
Computation and Language
AI summaryⓘ
The authors focus on diffusion language models that generate text step-by-step by gradually refining guesses for each word. They point out that most current methods only look at the latest guess, which can cause the model to jump to a word on shaky ground instead of waiting for stronger, consistent signals. To fix this, they introduce Trajectory-Aware Commit Gating (TACG), a way to decide when a word’s guess is stable enough to lock in, using past prediction information without needing extra training. Their tests show that TACG can keep or improve accuracy while making the generation process more efficient across several coding and math tasks.
Diffusion Language ModelsToken CommitmentDenoisingLogitsTemporal Implicit Logits GuidanceHistory GateExponential Moving AverageLanguage Model DecodingCode Generation BenchmarksMath Problem Benchmarks
Authors
Chengcheng Wang, Tingzhang Luo, Wenhao Li, Jianyuan Guo, Chang Xu
Abstract
Diffusion language models (DLLMs) generate text by iteratively denoising masked positions, exposing a trajectory of predictive distributions rather than a single instantaneous belief. Most existing decoders ignore this trajectory and commit tokens from the current snapshot alone, conflating confidence with commitment readiness: a transient top-1 peak under incomplete context can be locked in, while candidates with consistent cross-step support are delayed. We propose Trajectory-Aware Commit Gating (TACG), a training-free gate-level decoder that anchors token identities to the base posterior and uses trajectory-aware signals only to decide whether the current proposal is ready to commit. TACG combines Temporal Implicit Logits Guidance (TILG), which keeps an exponential moving average of past logits as a self-reference and contrasts the current logits against this reference in natural-parameter space, with a History Gate (HG) that enforces short-term proposal persistence before commitment. Together with a capped extra-promotion budget, these components yield a stability-constrained commit rule without auxiliary networks or extra forward passes. We evaluate TACG on LLaDA, Dream, and LLaDA2-Mini across code (HumanEval, MBPP) and math (GSM8K, MATH500) benchmarks; it typically improves or preserves accuracy while reducing denoising steps and increasing tokens per forward (TPF). The code is publicly available at https://github.com/Clarence-CV/TACG-DLLM.