AI Agent Memory: Why Your Assistant Will Remember You

Language models do not remember you. Each request starts from whatever text is sent with it. Everything that feels like memory in ChatGPT, Claude or a custom agent is engineering around the model: deciding what to write down, where to keep it and when to read it back. That engineering is moving fast, and it changes what assistants can do and what they know about you.

What is AI agent memory?

AI agent memory is any information an agent keeps outside a single model call and reuses later. The model's context window is not memory in this sense. It is closer to a desk: whatever is on it right now is visible, and it is cleared when the conversation ends or gets too long.

Real memory needs three things the model does not provide by itself:

• A store: files, a database or a vector index that survives between sessions.
• A write policy: rules for what is worth saving and what is not.
• A read policy: rules for when to look things up and how much to load.

The most useful framework I know for this comes from a 2023 paper, Cognitive Architectures for Language Agents (CoALA), by Theodore Sumers, Shunyu Yao, Karthik Narasimhan and Thomas Griffiths. It borrows terms from cognitive science and applies them to agents built on language models. I will use plainer words for the same ideas.

The three kinds of memory, in plain language

An agent needs a memory for today, a memory for facts and a memory for skills, and each is built differently. CoALA defines working memory as the information held "for the current decision cycle", episodic memory as stored experience from earlier cycles, semantic memory as an agent's knowledge "about the world and itself", and procedural memory as both knowledge in the model's weights and instructions written in the agent's code.

The third row is the one people overlook. A file of standing instructions that a coding agent reads at the start of every session is procedural memory. I wrote about one form of it in how to write a CLAUDE.md, and the same idea applies to any agent that repeats a kind of work.

How do ChatGPT and Claude remember you today?

Consumer assistants now keep a long-term memory by default, and developer APIs let you build your own. As of late 2026, the two approaches look like this.

In the Claude apps, memory is saved as you chat rather than summarised after the conversation ends. Anthropic's help page says it is on by default for Free, Pro, Max and Ultra plans, that each project has its own memory, and that you can view, edit or delete individual topics in settings. Pausing memory stops Claude using or adding to it, resetting deletes everything, and incognito chats are not saved to memory at all.

ChatGPT has had a long-term memory feature since at least 2024. Its history includes a useful warning, which I come back to below.

For developers, Anthropic's API offers a memory tool that lets the model create, read, update and delete files in a /memories directory. The important design choice is that it runs client-side: the model only requests operations, and your application decides where the data lives, whether a per-user folder or rows in a database. The documentation pairs it with compaction for long-running agents, so a summarised conversation does not lose what must survive.

That client-side pattern is the one I recommend for client work. The memory sits in infrastructure you already secure, under the same retention rules as the rest of your data.

Why memory is the next big shift for agents

Memory turns an assistant that answers questions into one that can carry work across days. Without it, every session repeats the same discovery: who you are, what the project is, what was decided last week. (For coding agents, I cover how to cut that repeated briefing in context engineering for unfamiliar codebases.) With it, an agent can pick up where it stopped.

Researchers increasingly treat it as foundational. A 2025 survey of agent memory research, Memory in the Age of AI Agents, opens by stating that memory "has emerged, and will continue to remain, a core capability of foundation model-based agents." Anthropic's own documentation describes a pattern for multi-session software work where the first session writes a progress log and feature checklist, and each later session starts by reading them.

My view, as someone who builds agents for clients: memory is where most of the practical gains of the next few years will come from, more than raw model intelligence. A capable model that forgets your context is less useful at work than a slightly weaker one that remembers it. This matters even more once every employee has an AI agent, because an agent that knows how your team works is worth far more than one you brief from scratch every morning.

Memory also has a retrieval problem. Saving everything is easy; finding the right three facts at the right moment is hard. That is the same problem agentic RAG solves for documents, and the techniques overlap.

What privacy and security risks does AI memory create?

An agent that remembers you holds a growing profile of you, and anything that can write to that profile can change how the agent behaves later. Three risks deserve attention.

1. Memory can be poisoned. In 2024 the security researcher Johann Rehberger showed that a prompt injection could plant instructions in ChatGPT's memory, enabling "continuous data exfiltration of any information the user typed or responses received by ChatGPT", including in future sessions. OpenAI fixed that exfiltration route in the macOS app, and Rehberger's advice was to check stored memories regularly and remove anything suspicious. The general lesson: content an agent reads from a web page or document must never be able to write directly to long-term memory.

2. Memory accumulates sensitive data. Anthropic's documentation notes that Claude usually refuses to write sensitive information to memory, but recommends that developers add validation that strips sensitive data for stronger guarantees, cap file sizes and periodically delete memory that has not been used in a long time.

3. Memory storage is an attack surface. The same documentation warns that a path such as /memories/../../secrets.env could reach files outside the memory folder, so every path must be validated. It is ordinary web security, applied to a new feature.

How I design memory for client agents

On client projects I treat memory as a product feature with rules, not a hidden cache. My checklist:

• Make it visible. Users should be able to see what the agent remembers about them and delete it, as the consumer apps now allow.
• Scope it. Separate memory per user and per project, so one customer's details never surface in another's session.
• Gate the write path. Only the user's own instructions or verified system events can create memories. Text from web pages, emails or uploaded files cannot.
• Never store secrets. API keys, passwords and payment details are filtered before anything is written.
• Let it expire. Unused memories age out, which keeps retrieval sharp and reduces what could leak.
• Test recall. I check that the agent retrieves the right memory for a task and ignores stale ones, the same way I evaluate any agent or retrieval system.

Work with me

I build agents that remember the right things, with memory you can inspect, scope and delete. See AI agent development and AI chatbot development, or browse my work and book a scoping call.