Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarios
2026-04-10 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how big language models perform differently depending on the task and cost, so they created a system to pick the best model for each situation. They noticed existing solutions don't work well when there's no training data for a new task, so they made a new method that generates example questions and answers to simulate these new tasks. Using this, they built TRouter, which understands different task types and predicts both the cost and quality of answers to make smarter model choices. Their tests show that this approach works better both when starting fresh and with known tasks.
large language modelsmodel routingcold-start problemtask taxonomydata synthesisquery distributioncost-performance trade-offlatent variablestask-type-aware routingbenchmark evaluation
Authors
Hui Liu, Bin Zou, Kecheng Chen, Jie Liu, Wenya Wang, Haoliang Li
Abstract
Large language models (LLMs) exhibit substantial variability in performance and computational cost across tasks and queries, motivating routing systems that select models to meet user-specific cost-performance trade-offs. However, existing routers generalize poorly in cold-start scenarios where in-domain training data is unavailable. We address this limitation with a multi-level task-profile-guided data synthesis framework that constructs a hierarchical task taxonomy and produces diverse question-answer pairs to approximate the test-time query distribution. Building on this, we introduce TRouter, a task-type-aware router approach that models query-conditioned cost and performance via latent task-type variables, with prior regularization derived from the synthesized task taxonomy. This design enhances TRouter's routing utility under both cold-start and in-domain settings. Across multiple benchmarks, we show that our synthesis framework alleviates cold-start issues and that TRouter delivers effective LLM routing.