Artifacts as Memory Beyond the Agent Boundary
2026-04-09 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors explore how an agent's environment can act like an external memory to help it learn better in Reinforcement Learning tasks. They introduce a way to mathematically describe how parts of the environment, called artifacts, can hold important information that reduces the need for the agent to remember everything internally. Their experiments show that when agents can see paths in their environment, they don't need as much memory to make good decisions. This effect happens naturally through what the agent observes, without extra programming. The authors suggest that future research could find ways to use the environment to replace some internal memory in intelligent agents.
Reinforcement LearningSituated CognitionExternal MemoryAgentEnvironmentArtifactsPolicy LearningSpatial PathsSensory Stream
Authors
John D. Martin, Fraser Mince, Esra'a Saleh, Amy Pajak
Abstract
The situated view of cognition holds that intelligent behavior depends not only on internal memory, but on an agent's active use of environmental resources. Here, we begin formalizing this intuition within Reinforcement Learning (RL). We introduce a mathematical framing for how the environment can functionally serve as an agent's memory, and prove that certain observations, which we call artifacts, can reduce the information needed to represent history. We corroborate our theory with experiments showing that when agents observe spatial paths, the amount of memory required to learn a performant policy is reduced. Interestingly, this effect arises unintentionally, and implicitly through the agent's sensory stream. We discuss the implications of our findings, and show they satisfy qualitative properties previously used to ground accounts of external memory. Moving forward, we anticipate further work on this subject could reveal principled ways to exploit the environment as a substitute for explicit internal memory.