Explainable Reinforcement Learning via Physics-Aware Policy Distillation
2026-07-27 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how to make complicated AI controllers used in robots and cars easier to understand. They take a powerful but hard-to-understand AI called TD3 and create a simpler model using decision trees that can copy its behavior. By adding smart features and special data methods, the simpler model performs just as well. They also find that changing from smooth controls to rule-based controls causes certain predictable switching behaviors but still keeps the system stable and safe. This work helps make AI decisions more transparent without losing performance.
Deep Reinforcement LearningTwin Delayed DDPG (TD3)Policy DistillationDecision TreesInverted PendulumBang-Bang ControlBIBO StabilityContinuous ControlSurrogate ModelsInterpretability
Authors
Shaker Al-Tamari, Waled Kadour
Abstract
In safety-critical sectors such as robotics and automotive engineering, the deployment of Deep Reinforcement Learning (DRL) is often hindered by the black-box nature of deep neural networks. This lack of transparency poses significant challenges for regulatory compliance and human-agent trust. This paper presents an experimental study aimed at making high-performance continuous control DRL systems interpretable. A policy distillation framework is implemented using the classic Inverted Pendulum benchmark. A high-performance Twin Delayed DDPG (TD3) agent serves as an opaque, continuous teacher model, whose policy is distilled into an interpretable student surrogate based on a shallow Decision Tree. By leveraging a custom physics-aware feature and "Noisy Oracle Rollouts" for dataset generation, the distillation process achieves performance equivalent to the expert teacher. Furthermore, comparative control theory analysis reveals a fundamental trade-off: transitioning from continuous to discrete rule-based control induces high-frequency Bang-Bang actuation and a stable bimodal limit cycle. Simulation results indicate that Bounded-Input Bounded-Output (BIBO) stability is maintained while providing both global and local interpretability for safe autonomous systems.