AI summaryⓘ
The authors developed Sentinel-RL, a system to help detect and respond to cybersecurity threats in large networks more reliably than typical large language models (LLMs). They combined a graph-based method to summarize network activity with a reinforcement learning policy that picks investigation actions, while the LLM only creates readable reports for humans to approve. Their tests showed the system quickly processes huge data sets, detects threats accurately, and completes response cycles in just seconds. They also shared practical engineering solutions for running this at scale and discussed how it fits enterprise security needs. Overall, the authors separated reasoning about network structure from language generation to improve efficiency and reliability.
Large Language ModelSecurity Operations Center (SOC)Proximal Policy Optimization (PPO)Graph Attention EncoderAuthentication GraphReinforcement LearningNeo4jCybersecurity AlertsRed Team EventsHuman-in-the-Loop
Abstract
Large language model (LLM) agents are increasingly proposed as autonomous SOC analysts, but two limitations make them unreliable at enterprise scale: a finite context window cannot hold a multi-thousand-host authentication graph, and free-form generation offers no guarantee that a recommended containment action is consistent with the topology it operates on. We present Sentinel-RL, an agentic-SOC architecture that decouples topological reasoning from semantic reasoning: a heterogeneous graph attention encoder summarizes the live authentication subgraph into a fixed-dimensional state, a Proximal Policy Optimization (PPO) policy maps this state to a constrained set of investigative actions, and an LLM agent loop is restricted to consuming the policy's recommendations and producing analyst-readable narratives gated by a critic. We instantiate the system on the LANL Comprehensive, Multi-Source Cyber-Security Events dataset and the Indiana University Quartz HPC cluster, reporting four results: (i) a two-phase CREATE ingestion pattern loads a 24M-edge authentication subgraph into Neo4j in 14.2 minutes on a single 32-core node, roughly 24x faster than the canonical MERGE-based pipeline; (ii) a sliding-window alert engine reliably trips a 25-event/10-second threshold in <=2.5 s across 50 trials; (iii) PPO training over 200 iterations converges to a mean episodic return of 8.74+/-0.31, with held-out precision of 0.91 and recall of 0.87 on labeled red-team events; and (iv) the integrated containment loop completes a full detect-investigate-recommend-human-approve cycle in a median of 6.3 s. We contribute a reusable engineering pattern (the hot-node deadlock workaround), a portable HPC deployment pattern (anchor-node co-location), and an enterprise-readiness analysis covering false-positive economics, reversibility guarantees, audit compliance, and the human-approval boundary.