FORGE: Multi-Agent Graduated Exploitation and Detection Engineering
2026-06-02 • Cryptography and Security
Cryptography and SecurityArtificial IntelligenceMultiagent Systems
AI summaryⓘ
The authors developed FORGE, a system that helps three normally separate research areas about software vulnerabilities work together. FORGE uses five agents to create vulnerable apps, try exploiting them step-by-step, and then make detection rules from the exploitation data. Their method produces different levels of exploitation success, which helps improve both how vulnerabilities are prioritized and how detection rules are generated. Testing on over 600 vulnerabilities showed the system can reliably exploit many types and create accurate detection rules with few false alarms. The authors also found that exploitation success doesn’t depend much on common vulnerability scoring methods.
vulnerability disclosureexploit generationvulnerability prioritizationdetection rule engineeringCVE metadatamulti-agent systemSigma rulesSnort rulesOpenTelemetryLLM oracle
Authors
Farooq Shaikh
Abstract
Vulnerability disclosure volumes now far exceed organizational assessment capacity, yet three adjacent research communities (proof-of-concept generation, vulnerability prioritization, and detection rule engineering) operate largely in isolation. Existing automated exploit generation systems report binary pass/fail outcomes, discarding partial progress and producing no signal for the other two communities. This paper presents FORGE, a multi-agent system that bridges these three silos through graduated exploitation depth. Five specialized agents (Intel, Generator, Planner, Exploit, and Detector) execute in a fixed pipeline that (1) generates targeted vulnerable applications from CVE metadata, (2) conducts coached, multi-turn exploitation assessed by an LLM-primary oracle on a four-level taxonomy (L0: no evidence through L3: full compromise), and (3) produces Sigma and Snort detection rules grounded in OpenTelemetry exploitation traces. Graduated depth is the bridging mechanism: deeper exploitation yields richer behavioral traces for detection engineering, while depth data across scoring bands provides ground truth for prioritization validation. A tiered knowledge architecture accumulates intelligence across assessments, transferring build and exploitation experience to subsequent CVEs. Evaluation on 603 CVEs from the CVE-GENIE dataset achieves 67.8% end-to-end L1+ exploitation at USD 1.50 per CVE across eight languages and 187 CWE types. Exploitation rates remain near 68% regardless of EPSS or CVSS band, indicating that pattern-level reachability is orthogonal to metadata-based prioritization. Detection rules from L2+ exploitation achieve significantly higher span-normalized grounding than L1-derived rules (p=0.035), and 93.4% of generated Snort rules produce zero false positives against a synthetic benign corpus.