Enhancing the MADDPG Algorithm for Multi-Agent Learning via Action Inference and Importance Sampling
2026-06-03 • Machine Learning
Machine Learning
AI summaryⓘ
The authors improved an existing multi-agent reinforcement learning algorithm called MADDPG by adding two features. First, their new Action Inference method helps each agent guess what other agents will do, making learning more stable and cooperative. Second, they used a special sampling technique to focus on recent important experiences, which helps agents learn better in changing environments. They tested their ideas on a Predator-Prey game and found both changes boosted performance and exploration efficiency.
multi-agent reinforcement learningMADDPGAction Inferenceimportance samplinggeometric distributionreplay buffernon-stationaritypredator-prey taskPettingZoopolicy stability
Authors
Marc Walden, Jason Liu, Shaashwath Sivakumar, Ryan Liu, Hamza Khan
Abstract
We investigate multi-agent deep reinforcement learning and propose two enhancements to the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. First, we introduce a novel Action Inference mechanism that enables each agent to predict other agents' intended actions, thereby improving the accuracy and stability of its own policy. Second, we apply an importance sampling strategy, using geometric distribution, in the replay buffer to prioritize more recent and informative experiences, which helps mitigate the non-stationarity inherent in multi-agent environments. We evaluate both modifications on the discrete-action Predator-Prey task provided by the PettingZoo library, a flexible Python interface for general multi-agent reinforcement learning benchmarks. Our results indicate that Action Inference is effective in improving learning stability and inter-agent cooperation and that importance sampling using geometric distribution can lead to significant improvements in exploration efficiency over standard MADDPG. Code available at https://github.com/shaashwathsivakumar/MARL_Proj