ASPECT:Analogical Semantic Policy Execution via Language Conditioned Transfer
2026-04-09 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors address a common problem where reinforcement learning agents find it hard to apply what they've learned to new but similar tasks. Instead of using fixed categories to recognize tasks, they use natural language and a special model to translate new situations into descriptions the agent understands. By using a large language model to reinterpret observations during testing, their method helps the agent imagine states it was trained on, allowing it to reuse its skills without extra learning. This approach enables the agent to handle a wider variety of new tasks without relying on fixed labels.
Reinforcement LearningZero-shot TransferVariational Autoencoder (VAE)Large Language Model (LLM)Natural Language ConditioningSemantic MappingPolicy ReuseTask GeneralizationZero-shot Learning
Authors
Ajsal Shereef Palattuparambil, Thommen George Karimpanal, Santu Rana
Abstract
Reinforcement Learning (RL) agents often struggle to generalize knowledge to new tasks, even those structurally similar to ones they have mastered. Although recent approaches have attempted to mitigate this issue via zero-shot transfer, they are often constrained by predefined, discrete class systems, limiting their adaptability to novel or compositional task variations. We propose a significantly more generalized approach, replacing discrete latent variables with natural language conditioning via a text-conditioned Variational Autoencoder (VAE). Our core innovation utilizes a Large Language Model (LLM) as a dynamic \textit{semantic operator} at test time. Rather than relying on rigid rules, our agent queries the LLM to semantically remap the description of the current observation to align with the source task. This source-aligned caption conditions the VAE to generate an imagined state compatible with the agent's original training, enabling direct policy reuse. By harnessing the flexible reasoning capabilities of LLMs, our approach achieves zero-shot transfer across a broad spectrum of complex and truly novel analogous tasks, moving beyond the limitations of fixed category mappings. Code and videos are available \href{https://anonymous.4open.science/r/ASPECT-85C3/}{here}.