TiCo: Time-Controllable Training for Spoken Dialogue Models
2026-03-23 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors created TiCo, a method that helps spoken dialogue models control how long their responses are, like making answers last about 15 seconds. Normally, these models can talk naturally but can't keep track of time well. TiCo works by teaching the model to use special time markers while it talks, so it knows how long it's been speaking and can adjust the response to fit the desired length. The method is simple, needs little extra data, and uses reinforcement learning. Tests showed TiCo makes models better at matching time limits without making the responses worse.
spoken dialogue modelsresponse duration controltime-aware generationspoken time markerspost-training methodsreinforcement learningvoice assistantsinteractive agentsnatural language generation
Authors
Kai-Wei Chang, Wei-Chih Chen, En-Pei Hu, Hung-yi Lee, James Glass
Abstract
We propose TiCo, a simple post-training method for enabling spoken dialogue models (SDMs) to follow time-constrained instructions and generate responses with controllable duration. This capability is valuable for real-world spoken language systems such as voice assistants and interactive agents, where controlling response duration can improve interaction quality. However, despite their strong ability to generate natural spoken responses, existing models lack time awareness and struggle to follow duration-related instructions (e.g., "Please generate a response lasting about 15 seconds"). Through an empirical evaluation of both open-source and commercial SDMs, we show that they frequently fail to satisfy such time-control requirements. TiCo addresses this limitation by enabling models to estimate elapsed speaking time during generation through Spoken Time Markers (STM) (e.g., <10.6 seconds>). These markers help the model maintain awareness of time and adjust the remaining content to meet the target duration. TiCo is simple and efficient: it requires only a small amount of data and no additional question-answer pairs, relying instead on self-generation and reinforcement learning. Experimental results show that TiCo significantly improves adherence to duration constraints while preserving response quality.