MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
2026-07-01 • Information Retrieval
Information RetrievalArtificial Intelligence
AI summaryⓘ
The authors explain that memory helps AI agents work better over time, but sometimes using memory can make them agree too much with users, even if it's not accurate. They point out that current tests mostly check if memory is stored or retrieved correctly, but don't check if the memory affects the agent's decisions properly. To fix this, the authors created MemSyco-Bench, a new set of tests that see if AI agents can tell when to trust memories, ignore wrong ones, and use memories correctly in different situations. They also made their test materials available for others to use.
LLM-based agentsmemory retrievalsycophancyfactual accuracydecision-makingmemory benchmarkspersonalizationmemory updatesobjective evidenceMemSyco-Bench
Authors
Zhishang Xiang, Zerui Chen, Yunbo Tang, Zhimin Wei, Ruqin Ning, Yujie Lin, Qinggang Zhang, Jinsong Su
Abstract
Memory has emerged as a cornerstone of modern LLM-based agents, supporting their evolution from single-turn assistants to long-term collaborators. However, memory is not always beneficial: retrieved memories often induce a critical issue of sycophancy, causing agents to over-align with the user at the cost of factual accuracy or objective reasoning. Despite this emerging risk, existing memory benchmarks primarily evaluate whether memories are correctly stored, retrieved, or updated, while overlooking how retrieved memories influence downstream reasoning and decision-making. To bridge this gap, we propose MemSyco-Bench, a comprehensive benchmark for evaluating memory-induced sycophancy in agent systems. MemSyco-Bench measures when memory should influence a decision and how valid memory should be used. Specifically, it covers five tasks that assess whether agents can reject memory as factual evidence, respect its applicable scope, resolve conflicts between memory and objective evidence, track memory updates, and use valid memory for personalization. All related resources are collected for the community at https://github.com/XMUDeepLIT/MemSyco-Bench.