Structured Thoughts For Improved Reasoning And Context Pruning
2026-07-11 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors created a new method called Structured Thoughts to help language models think more clearly and use less memory. They split the reasoning process into two parts: <try>, where the model explores ideas, and <outcome>, where it sums up the results. By training models on this format, they improved reasoning accuracy and made it possible to remove some of the less important information during thinking, saving memory. Their approach cuts memory use significantly but with a small drop in performance on math problems.
large language modelsreasoning chainsstructured reasoningfine-tuningcontext pruningmemory efficiencyscratch worksummarizationmathematical reasoningprompting
Authors
Zain Sarwar, Supriyo Chakraborty, Berkcan Kapusuzoglu, Chia-Hsuan Lee, Anirban Das, Stephen Rawls, Kartik Balasubramaniam, Sambit Sahu
Abstract
Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured Thoughts, a framework that organizes reasoning into alternating <try> and <outcome> blocks: <try> captures exploratory scratch work, while <outcome> contains the distilled conclusion of that step. We construct a dataset of structured thoughts by segmenting reasoning traces into <try> blocks and prompting an LLM to summarize each step into its corresponding <outcome>. Fine-tuning pretrained foundation models on this reformatted data produces models that adopt the structured reasoning style, leading to performance gains of up to 8.08\% on reasoning benchmarks compared to standard SFT. The explicit structure also enables context pruning: after each <try>/<outcome> pair, the <try> can be pruned, allowing the model to retain conclusions without keeping the full scratch work in the context. A proof-of-concept pruning implementation achieves an average of 85\% memory / context savings with an 8.67\% performance drop across mathematical tasks.