Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning
2026-06-02 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors present a new method called Agentic Chain-of-Thought Steering (ACTS) to make large language models think more efficiently. Instead of letting the model think freely, ACTS uses a controller that guides the model step-by-step based on a limited 'thinking budget.' This controller decides how the model should reason next, balancing accuracy and efficiency. Their experiments show that ACTS keeps performance high while using fewer steps, and it works across different tasks and models.
large language modelschain-of-thought reasoningMarkov decision processreinforcement learninginference-time controltoken efficiencyreasoning strategiesreward shapingcontroller agent
Authors
Yu Xia, Zhouhang Xie, Xin Xu, Byungkyu Kang, Prarit Lamba, Xiang Gao, Julian McAuley
Abstract
Large language models improve final-answer accuracy through extended chain-of-thought reasoning, but often spend tokens inefficiently and offer little inference-time control. Existing efficient reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates reasoning steering as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the reasoning trace and remaining thinking budget, then issues a steering action consisting of a reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient reasoning while preserving the reasoner's generation continuity. We initialize the controller agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple benchmarks show that ACTS matches full-thinking performance with substantial token savings, and enables controllable accuracy-efficiency trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS.