Do LLMs Benefit From Their Own Words?
2026-02-27 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors studied whether large language models really need to remember their own previous replies in a conversation. They found that often, removing the model’s past responses from the chat history didn’t lower the quality of its answers and actually made the conversation easier to handle. Sometimes, including past responses caused mistakes to build up. Based on this, the authors suggest that skipping some past model replies can save memory and sometimes improve answers.
large language modelsmulti-turn conversationcontext lengthpromptingcontext pollutionhallucinationsresponse qualitymemory consumptionself-contained promptscontext filtering
Authors
Jenny Y. Huang, Leshem Choshen, Ramon Astudillo, Tamara Broderick, Jacob Andreas
Abstract
Multi-turn interactions with large language models typically retain the assistant's own past responses in the conversation history. In this work, we revisit this design choice by asking whether large language models benefit from conditioning on their own prior responses. Using in-the-wild, multi-turn conversations, we compare standard (full-context) prompting with a user-turn-only prompting approach that omits all previous assistant responses, across three open reasoning models and one state-of-the-art model. To our surprise, we find that removing prior assistant responses does not affect response quality on a large fraction of turns. Omitting assistant-side history can reduce cumulative context lengths by up to 10x. To explain this result, we find that multi-turn conversations consist of a substantial proportion (36.4%) of self-contained prompts, and that many follow-up prompts provide sufficient instruction to be answered using only the current user turn and prior user turns. When analyzing cases where user-turn-only prompting substantially outperforms full context, we identify instances of context pollution, in which models over-condition on their previous responses, introducing errors, hallucinations, or stylistic artifacts that propagate across turns. Motivated by these findings, we design a context-filtering approach that selectively omits assistant-side context. Our findings suggest that selectively omitting assistant history can improve response quality while reducing memory consumption.