Learning While Acting: A Skill-Enhanced Test-Time Co-Evolution Framework for Online Lifelong Learning Agents
2026-06-03 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors address the challenge of teaching AI language agents to keep learning and improving while they work in changing environments. They created LifeSkill, a two-part learning method that first helps the AI find useful skills by checking how well these skills help solve tasks, and then lets the AI update itself continuously using feedback while it works. This approach helps the AI learn more effectively than older methods that just reuse past experiences without adapting. Tests showed that LifeSkill improved performance notably compared to previous lifelong learning agents.
Lifelong learningLarge Language ModelsReinforcement learningSkill extractionPolicy rolloutsOnline learningTest-time adaptationVerifier-guided learningSkill internalizationLifelongAgentBench
Authors
Bo Mao, Jie Zhou, Yutao Yang, Xin Li, Xian Wei, Qin Chen, Xingjiao Wu, Liang He
Abstract
Lifelong learning is essential for Large Language Model (LLM) agents operating in dynamic, interactive environments. However, existing lifelong learning agents for long-horizon tasks typically depend on discrete skill or past experiences retrieval with static parameters during inference, which prevents them from continuously internalizing test-time feedback like human learners. To bridge this gap, we propose Skill-enhanced Test-Time Co-Evolution (\texttt{LifeSkill}), a two-stage reinforcement learning framework for Online Lifelong Learning Agents. Specifically, we design Verifier-Guided Skill Learning that addresses the lack of direct supervision for skill extraction by rewarding candidate skills according to the average verifier success of multiple skill-conditioned policy rollouts, encouraging the model to generate skills that are useful for solving tasks rather than merely plausible in text. Furthermore, we introduce Online Skill Internalization, which continuously improves the policy model during test-time interaction by transforming skill-conditioned trajectories into reward signals. This enables the agent to directly internalize reasoning capabilities into its parameters, avoiding the context bloat of experience retrieval. Experiments on LifelongAgentBench show that LifeSkill improves average performance by 7 absolute points by comparing with existing lifelong agent baselines.