SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
2026-04-02 • Machine Learning
Machine Learning
AI summaryⓘ
The authors explore a method to teach language models skills by putting the knowledge inside the model itself, rather than just having it look up skills each time it needs them. They created SKILL0, which trains the model gradually, starting with lots of skill information and then slowly removing it, so the model learns to perform tasks on its own without extra help during use. Their experiments showed that SKILL0 improves performance on tasks while using fewer tokens, making the model more efficient. This approach helps models act independently without needing to fetch external skill information every time.
Large Language ModelsSkill AugmentationIn-context LearningReinforcement LearningCurriculum LearningZero-shot LearningTool Use in AIToken Efficiency
Authors
Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
Abstract
Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization. SKILL0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching he model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file's on-policy helpfulness, retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that SKILL0 achieves substantial improvements over the standard RL baseline (+9.7\% for ALFWorld and +6.6\% for Search-QA), while maintaining a highly efficient context of fewer than 0.5k tokens per step. Our code is available at https://github.com/ZJU-REAL/SkillZero.