Anchored Self-Play for Code Repair
2026-07-03 • Software Engineering
Software EngineeringComputation and Language
AI summaryⓘ
The authors study how to improve language models' ability to fix buggy code using a method called generator-fixer self-play, where the model learns by creating bugs and then fixing them. They built a new test called BugSourceBench with different types of bugs from human and AI sources. They found that self-play alone makes the model better at fixing synthetic bugs but worse on real human bugs. To fix this, they introduced Anchored Self-Play (ASP), which uses a small set of real bugs to keep the training focused. ASP improved fix rates across all bug types compared to standard self-play.
language modelscode repairself-playreinforcement learningbug fixingprogram synthesisbenchmarkBugSourceBenchcurriculum learningcode embedding
Authors
Caroline Choi, Zeyneb Kaya, Shirley Wu, Tengyu Ma, Tatsunori Hashimoto, Ludwig Schmidt
Abstract
Code repair is an important capability for language models (LMs): given a buggy program and unit tests, an LM must produce a fixed program that passes the tests. Because code repair data is limited, we aim to scale supervision by using an LM to generate bug--fix tasks. We propose __generator--fixer self-play__, in which a single model is trained with reinforcement learning to generate bugs and fix them. As the fixer improves, the generator adapts to produce more difficult bugs, yielding an automatic curriculum. To test whether this curriculum generalizes, we introduce BugSourceBench, a repair benchmark spanning realistic bug sources: bugs in human-written code, LM-generated code, and human-edited LM-generated code. On BugSourceBench, we find that self-play drifts toward difficult but unrealistic bugs, improving on synthetic bugs but degrading on human-authored ones. We propose Anchored Self-Play (ASP), which anchors self-play with a small reference set by adding a code-embedding similarity reward for generation and mixing reference bugs into fixer training. Across bug sources, ASP achieves the best fix rates, improving average fix rate over standard self-play by $+24\%$ relative / $+7.0$ pp absolute, with gains on bugs from both LMs and humans.