Trace Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems
2026-08-26 • Artificial Intelligence
Artificial IntelligenceComputation and Language
AI summaryⓘ
The authors explain that just getting the right answer from a language model isn’t enough to trust it, especially for tasks with structured data. They propose a new way to check if the steps behind an answer are clear, correct, and repeatable, which they call Trace Integrity. They show that many answers that seem right don’t have a valid reasoning trace behind them, meaning the system might be wrong without us knowing. Their work highlights the need to look at both the answer and the explanation process for better reliability in real-world applications.
Large Language ModelsTrace IntegrityStructured DataExecution ContractsBenchmark AccuracyOperator-FaithfulReplayabilityAuditable ComputationCAIT RateDeployment Reliability
Authors
Srimonti Dutta, Akshata Kishore Moharir
Abstract
Answer accuracy is an insufficient reliability signal for LLM data agents. In structured-data tasks, a benchmark-correct answer can be produced by an invalid trace. This paper introduces Trace Integrity, a deployment reliability criterion for evaluating whether the computation recorded behind an answer is explicit, executable, schema-valid, operator-faithful, replayable, answer-consistent, and auditable. We identify the Structure Gap as the deployment failure mode that makes Trace Integrity necessary: natural-language reasoning and free-form rationales do not reliably specify the operator-level programs required by real-world systems. We operationalize Trace Integrity with execution contracts, structured artifacts that bind user intent to schema elements, operator plans, assumptions, executable queries, verification status, and final-answer linkage. We also introduce CAIT (Correct Answer / Invalid Trace) Rate, which measures how often answer-only evaluation counts computationally unsupported outputs as successes. In an empirical demonstration on BIRD Mini-Dev, Direct SQL, Operation Summary + SQL, and Contract-First SQL achieve answer accuracies of 20%, 22%, and 24%, while their Trace Integrity Pass Rates are 39%, 43%, and 40% and their CAIT Rates remain high at 55%, 59.1%, and 45.8%, showing that answer accuracy, trace validity, and silent-failure risk are distinct evaluation signals. Real-world LLM data agents should, therefore, be evaluated not only by whether their outputs match a reference answer, but by whether those outputs are backed by auditable computation.