AI summaryⓘ
The authors study a type of attack on AI systems that use external documents to answer questions, called Agentic RAG systems. These systems usually resist attacks because they reason step-by-step and ignore irrelevant harmful documents. However, the authors show that attackers can still trick these systems by publishing specially crafted documents that guide the system’s reasoning in harmful ways, even without knowing the system’s internal details. They introduce KidnapRAG, a method that uses three types of fake documents to hijack the AI’s reasoning chain and cause it to rely on bad information. Their experiments confirm KidnapRAG is more effective than previous attacks when the attacker has limited access.
Retrieval-Augmented GenerationAgentic RAG systemspoisoning attacksblack-box attackquery reformulationmulti-step reasoningLLM backbonesretrieval poisoningadversarial documentsKidnapRAG
Authors
Chanwoo Choi, Euntae Kim, Kyuho Lee, Youngsam Chun, Jinhee Jeong, Eunmi Kim, Myunggyo Oh, Junseo Jang, Buru Chang
Abstract
Retrieval-Augmented Generation (RAG) systems are vulnerable to poisoning attacks that inject malicious documents into the retrieval process to manipulate model outputs. Recent Agentic RAG systems are more robust to such attacks because they iteratively perform retrieval and reasoning, allowing them to ignore weakly relevant poisoned documents and preserve the reasoning chain induced by the user query. However, existing attacks on Agentic RAG systems often assume white-box access to system prompts, reasoning traces, retrievers, or model parameters, limiting their applicability in realistic settings. In this paper, we study black-box poisoning attacks against Agentic RAG systems, where the attacker can only publish externally retrievable poisoned documents. We propose KidnapRAG, a sequential poisoning attack that hijacks the agent's multi-step reasoning chain using three role-specific documents: Bait, Chain-Link, and Mal-Ins, which attract initial retrieval, induce query reformulation, and provide attacker-controlled evidence, respectively. Experiments across multiple Agentic RAG frameworks, LLM backbones, and benchmarks show that KidnapRAG consistently outperforms existing poisoning baselines under black-box conditions. Further analyses show that KidnapRAG progressively weakens the original retrieval intent, redirects retrieval behavior, and increases reliance on attacker-controlled evidence. Our code is publicly available at https://github.com/chanwoochoi316/KidnapRAG.