Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills
2026-07-24 • Computation and Language
Computation and Language
AI summaryⓘ
The authors describe a new training method for large language models called Skill Self-Play (Skill-SP). Instead of learning tasks in a narrow environment or relying on unreliable open-ended tasks, they use distinct skills that focus on specific scenarios and a controller that dynamically picks these skills. Their method involves three parts: one proposes tasks, another solves them, and a controller manages skills, all working together in a loop to improve over time. This approach balances reliable feedback with variety, helping models learn better across different challenges. Tests show it improves performance on tasks involving tool use and reasoning.
Large Language ModelsSelf-EvolutionReinforcement LearningSkill Self-PlayTask DiversityVerification ReliabilityDynamic Skill ControllerCo-Evolutionary FrameworkTool-Use BenchmarksReasoning Benchmarks
Authors
Siyuan Huang, Pengyu Cheng, Haotian Liu, Tao Chen, Yihao Liu, Jingwei Ni, Shijie Zhou, Ziyi Yang, Gangwei Jiang, Mengyu Zhou, Yu Cheng, Xiaoxi Jiang, Guanjun Jiang
Abstract
LLM training is shifting from manual design and annotation to interaction-driven self-evolution. However, existing self-evolutionary methods face a fundamental dilemma between task diversity and verification reliability: environment-bound methods obtain precise feedback but confine learning to narrow domains, while open-ended self-generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute the training loop. We identify agent skills as a powerful middle ground to reconcile this tension: each skill ensures deep, verifiable execution in a specific scenario, while dynamic routing across skills maintains open-ended task variety. Leveraging this insight, we introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a dynamic skill controller. Orchestrated via a reinforcement learning loop, these components co-evolve in a continuous self-play loop: the proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library. This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration. Empirical evaluations on tool-use and reasoning benchmarks demonstrate that Skill-SP, serving as a robust evolution engine, consistently pushes the performance ceiling of competent backbones while catalyzing striking turnarounds for initially misaligned models. Our code is available at https://github.com/Qwen-Applications/skill-self-play.