Scoped Verification for Reliable Long-Horizon Agentic Context Evolution under Distribution Shift
2026-07-10 • Artificial Intelligence
Artificial IntelligenceComputation and Language
AI summaryⓘ
The authors study how large language model (LLM) agents keep updating their system instructions over time while the rest of the system remains unchanged. They propose a method called GRACE that stores instructions as a graph, allowing easier checking and updating of parts instead of long flat text. Their experiments show GRACE improves the agent’s reliability significantly compared to traditional text-based updates and even beats a stronger baseline model. The authors highlight the need for structured instruction storage and effective updating to maintain reliable long-term agent behavior.
LLM agentsagentic contextsystem instructionssemantic graphcontext evolutionverificationincremental updatestelecom agent harnessdistribution shiftpass3 reliability
Authors
Dan C. Hsu, Luke Lu
Abstract
Deployed LLM agents rely on agentic context, the model-external textual control content assembled by an operational harness. In this work, the mutable component of that context is a persistent system-level instruction that is updated from operational experience while the model, tools, and harness remain fixed. Over long evolution horizons, flat-text maintenance makes verification increasingly difficult as accumulated instructions grow and interact. We propose Graph-Regularized Agentic Context Evolution (GRACE), which maintains the persistent instruction component as a typed semantic graph and validates proposed updates within the local typed neighborhoods of modified nodes. Accepted graph updates are reconstructed as incremental edits to the textual instruction checkpoint used at deployment. We evaluate GRACE within a fixed telecom agent harness derived from $τ^2$-bench under a controlled distribution-shift protocol. Across five independent replications, GRACE improves strict reliability, measured by pass^3, from the Gemini 2.5 Flash zero-shot value of 0.091 to 0.673$\pm$0.136 at the final checkpoint. This exceeds a Gemini 3.1 Pro zero-shot reference of 0.242 on the same held-out set, while the flat-text HCE baseline finishes at 0.191$\pm$0.051. These results identify two requirements for reliable long-horizon context evolution, a structural substrate that makes verification local and a consolidation mechanism that keeps accumulated instruction content usable.