Scaling Expert Feedback with Reflective Edit Propagation in Compositional Knowledge Bases
2026-06-03 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors created a system called RAID to help keep specialized knowledge bases accurate and up to date. Instead of experts fixing entries one by one, RAID learns the reason behind a single expert's correction and applies that fix across the whole knowledge base automatically. This approach can save time and make it easier to maintain expert knowledge at scale. They tested RAID with public and private data and found it can effectively understand expert intentions and improve knowledge base updates.
Knowledge BaseLarge Language ModelExpert ValidationSemantic IntentReflective AgentKnowledge PropagationDomain-Specific KnowledgeUser StudyIntent InferenceAutomated Correction
Authors
Jiajing Guo, Xueming Li, Jorge Piazentin Ono, Wenbin He, Liu Ren
Abstract
Domain-specific knowledge bases (KBs) encode vertical expertise and proprietary information that organizations depend on, but curating them at scale is a persistent challenge. Although Large Language Models (LLMs) can draft initial entries efficiently, technical accuracy still requires human expert validation, and reviewing entries one by one at scale is impractical. We present Reflective Agent for Identifier Dictionary (RAID), a novel system that transforms individual expert edits into systematic knowledge updates. Unlike traditional "correct-and-save" paradigms, RAID utilizes a reflective agent to infer the underlying semantic intent behind a single expert edit and propagates that correction across the entire KB through a three-step architecture: Intent Inference, Reflection-based Planning, and User Controlled Execution. We evaluated the reflection and propagation performance on a public dataset and conducted a user study with subject matter experts with proprietary data. The evaluation shows RAID's technical feasibility in capturing expert intent and its potential to scale specialized expertise across industrial knowledge bases.