The 4 Types of AI Agent Memory Explained

Understanding how Artificial Intelligence agents process and retain information is crucial for developing more sophisticated and capable AI systems. Martin Keen, a Master Inventor at IBM, breaks down the four essential types of memory that AI agents need to function effectively. These memory types, inspired by human cognition, range from immediate contextual awareness to learned skills and past experiences.

Keen begins by drawing parallels to human memory, highlighting four key categories: short-term memory, factual knowledge, learned skills, and personal experience. He then maps these concepts onto AI agents, introducing a framework developed by a Princeton research team called Cola (Cognitive Architectures for Language Agents). This framework outlines four distinct types of memory crucial for AI agents.

Working Memory: The Agent's Scratchpad

The first type of memory is working memory, which corresponds to an AI agent's context window. This memory holds all the information the agent can currently access, including ongoing conversations, system instructions, and any loaded data or files. Keen likens this to Random Access Memory (RAM) in computers—fast and immediately accessible, but volatile and limited in size. While modern context windows can be quite large, they still have a ceiling, and performance can degrade as more information is crammed into them, making older or less relevant data harder for the model to access.

The full discussion can be found on IBM's YouTube channel.

Semantic Memory: The Knowledge Base

Semantic memory serves as the agent's knowledge base, storing facts, rules, and conventions. In academic circles, this is often implemented using vector databases or knowledge graphs. However, in many production AI systems, semantic memory is more straightforward, often residing in simple markdown files. Keen uses the example of a file named "cloud.md" which might contain project architecture, coding conventions, and build commands. By loading such files into the context window, agents gain access to essential, persistent knowledge, preventing them from repeating mistakes due to a lack of foundational understanding.

Procedural Memory: Knowing How to Do Things

Procedural memory dictates how an agent performs tasks. This is typically managed through structured files, such as the "skill.md" format mentioned by Keen. A skill is essentially a folder containing a markdown file that describes the skill, its function, and step-by-step instructions for execution. These skills can range from creating presentations to reviewing code. To manage computational resources, agents often use a progressive disclosure method, loading only a lightweight index of available skills initially. When a task aligns with a specific skill, the agent then retrieves the detailed instructions and any necessary supporting files or scripts.

Episodic Memory: Remembering Past Experiences

The fourth type, episodic memory, acts as a record of an agent’s past interactions and decisions. While a basic implementation might involve storing entire conversation transcripts, more advanced systems distill this information into concise notes. The agent selectively remembers what is likely to be useful in future interactions, effectively learning from its experiences. Keen emphasizes that this type of memory is crucial for an agent's improvement over time, allowing it to avoid repeating errors and to recall specific preferences or past outcomes. He notes that managing episodic memory, particularly deciding what information to discard as obsolete, is a significant engineering challenge.

Memory Needs Vary by Agent Complexity

Keen illustrates how different types of AI agents require varying combinations of these memory types. A simple reflex agent, like a thermostat or a basic router, might only need working memory. A slightly more complex agent, such as a password-reset chatbot, would likely need working memory and procedural memory to recall the specific skill. However, more advanced agents, like coding assistants, would benefit from all four types of memory—working memory for immediate context, semantic memory for project knowledge, procedural memory for coding skills, and episodic memory for past development experiences.

Ultimately, the effective use of these memory types distinguishes sophisticated AI agents from simple chatbots. While chatbots provide immediate responses, agents can offer more contextually relevant and intelligent interactions by drawing upon a rich tapestry of stored knowledge and learned experiences.