EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
2026-07-02 • Artificial Intelligence
Artificial IntelligenceComputation and Language
AI summaryⓘ
The authors created a new way to test how well computer programs, called autonomous agents, can improve their decision-making step by step using feedback. They designed a system called EvoPolicyGym with small tasks where the agents have a limited amount of chances to change and improve their strategies. In tests, a model named GPT-5.5 performed best overall. The authors also looked closely at how agents use their chances and turn feedback into improvements, finding that success depends on not just winning tasks, but also figuring out the right approaches and refining them carefully under limited feedback.
Autonomous agentsExecutable policyPolicy improvementReinforcement learningEvoPolicyGymInteraction budgetParametric tuningTrajectory diagnosticsGPT-5.5Benchmark evaluation
Authors
Zhilin Wang, Han Song, Runzhe Zhan, Jusen Du, Jiacheng Chen, Tianle Li, Qingyu Yin, Yulun Wu, Zhennan Shen, Tong Zhu, Yanshu Li, Guanjie Chen, Derek F. Wong, Yafu Li, Yu Cheng, Yang Yang
Abstract
Autonomous agents are increasingly expected to improve executable policies through feedback, yet existing evaluations often collapse this process into a final score or confound it with open-ended software-engineering progress. We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget. We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments that evaluates how agents iteratively improve explored policies. On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments. Beyond leaderboard results, EvoPolicyGym also provides trajectory-level diagnostics that distinguish how agents allocate budget, convert feedback into parametric tuning. These analyses show that strong autonomous policy evolution depends not only on isolated task wins, but on discovering task-appropriate mechanisms and refining policies under bounded feedback.