MemoryWAM: Efficient World Action Modeling with Persistent Memory
2026-06-18 • Robotics
Robotics
AI summaryⓘ
The authors developed MemoryWAM, a new model to help robots remember and understand both recent and older observations when manipulating objects. Existing models either remember only recent information or use a lot of resources to keep track of long histories, but MemoryWAM uses a smart memory system combining recent frames, special key frames, and summary tokens. This helps the robot quickly access detailed and compressed memories without using too much computing power. Tests show that MemoryWAM works better on tasks that need long-term memory while being efficient.
world action modelsrobotic manipulationmemory modelingvisual foresightattention mechanismnon-Markovian environmentslong-term memoryGPU memory usagevision-language-action
Authors
Sizhe Yang, Juncheng Mu, Tianming Wei, Chenhao Lu, Xiaofan Li, Linning Xu, Zhengrong Xue, Zhecheng Yuan, Dahua Lin, Jiangmiao Pang, Huazhe Xu
Abstract
Robust robotic manipulation in the real world requires not only an understanding of the current observation, but also memory and dynamics modeling. World action models (WAMs) possess these capabilities by jointly modeling visual foresight and actions conditioned on both current and historical observations, making them a promising paradigm for robotic manipulation. However, existing WAMs face a fundamental trade-off: methods with efficient inference typically condition only on a bounded window of recent observations and therefore struggle in non-Markovian environments, whereas methods that preserve long histories incur time and space costs that grow substantially with sequence length. To address this challenge, we introduce MemoryWAM, a world action model with efficient persistent memory. MemoryWAM uses a hybrid memory design that combines recent frames, event-boundary anchor frames, and compact gist tokens that summarize long-range history. A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context, supporting memory-dependent decision-making with reduced inference latency and GPU memory usage. Across long-horizon, memory-dependent manipulation tasks in both simulation and the real world, MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines while maintaining favorable computational efficiency.