Agent memory: why your AI has amnesia and what to do about it

You build something clever on a Tuesday afternoon. You and an AI agent work through a gnarly problem together — it learns your codebase, remembers the three approaches you already ruled out, picks up your habit of naming things a particular way. By five o'clock it feels less like a tool and more like a junior colleague who's finally got up to speed.

Then you close the laptop. On Wednesday morning you open it again, and you're talking to a stranger. It introduces itself and asks what you're working on. Every hard-won piece of shared context from the day before is simply gone.

That amnesia is the defining limitation of almost every AI agent in use today, not a glitch to be patched but a structural fact about how these systems work. And once you understand why it happens, the fix stops looking like magic and starts looking like a design decision you get to make.

The reason is simpler than it sounds

An AI model doesn't have a memory in the way you'd assume. What it has is a context window — a working set of text it can see right now, for this one exchange. Everything the model "knows" about your conversation lives in that window. When the session ends, the window empties. Nothing carries over unless something outside the model deliberately saves it and feeds it back in.

I've written before about how a single long conversation degrades as that window fills up — the context window problem, where the model slowly loses the thread even inside one session. Amnesia between sessions is the same limitation seen from a different angle. The window isn't just finite; it's temporary. It's working memory, and working memory resets.

Those earlier pieces were about one half of the problem: giving the model enough of your context to be useful. This is the other half: the agent building and keeping its own memory of the work as it goes. And that working set is only one kind of memory an agent needs. We've got several ourselves, and we use them without noticing which is which. An agent that has the first one and nothing else is always going to feel like it's meeting you for the first time.

The four kinds of memory

Think about what your Tuesday agent actually lost overnight. It forgot your naming conventions and which approaches you'd already ruled out. It forgot the whole shape of what you were doing together. Those are different kinds of forgetting, and it turns out there are four kinds of memory to match — the same four cognitive scientists use to describe our own, and a taxonomy that's become the common vocabulary across the agent frameworks. They map surprisingly well onto what you need when you're actually building something.

Working memory is the active context window: what the agent is holding in mind for the task right now. It's fast, it's rich, and it vanishes the moment the session ends. This is the one every agent already has.

Semantic memory is factual knowledge. Your company's products, your customers' names, the definitions and rules that don't change much. It's the difference between an assistant who knows your business and one you'd have to brief from scratch every time.

Episodic memory is the record of what happened. The conversation you had last Tuesday, the decision you reached, the thing that went wrong and how you fixed it. This is the one whose absence stings most, because it's what makes an agent feel like it has a history with you.

Procedural memory is how to do things — the repeatable workflows, the tool-use patterns, the "when this happens, do that" of a job done many times. It's the muscle memory an experienced employee has and a new hire doesn't.

Most agents you'll meet have working memory and nothing else. The interesting work — the work that turns a chatbot into something that compounds in value — is giving them the other three.

What actually works this week

You don't need a research budget to start. The solutions run from embarrassingly simple to genuinely sophisticated, and the simple end goes further than most people expect.

The simplest thing that works is a text file. When I build multi-agent systems, a lot of the memory is just plain-text files the agent reads at the start of a task and writes back to at the end. A file for stable facts about the business, a file for decisions made and why, a file for how I like things done. It's the same discipline as keeping your own context in a few markdown files — unglamorous, and it solves a genuinely large share of the problem. If your agent needs to remember your preferences and a handful of settled decisions, you may not need anything cleverer than this.

Where it breaks is scale. Once the agent has months of history, a single file it re-reads on every task turns into the context window problem all over again — too much material, most of it irrelevant to the job in hand.

That's where retrieval comes in. Instead of loading everything, you store your knowledge — documents, past chats, records — and pull back only the handful of pieces relevant to the question in front of the agent. Retrieval-augmented generation does this for semantic memory: the agent searches its knowledge and brings back what matters. A vector database does something similar for episodic recall, letting an agent find the three past interactions that resemble the current one rather than re-reading its entire history. Procedural memory is handled differently again: the workflows and tool-use rules an agent should follow live as saved routines, system prompts or checklists it loads for the job, so the sequence it worked out the hard way last month is the one it reaches for first this month.

Then there's a newer category: dedicated memory layers built specifically for this problem. Tools like Mem0 sit between your agent and its interactions, extracting what's worth remembering and serving it back when it's relevant. Letta, the project that grew out of the MemGPT research, treats the context window like an operating system treats RAM, paging information in and out of a persistent store so the agent's state survives long after any single session. You wire the agent to the memory layer and let it handle the remembering. Which of these tools ends up mattering, or whether the big platforms simply absorb the idea and make the category disappear, I wouldn't like to call yet — it's early, and this corner of the field is moving fast.

The right rung on that ladder depends on what you're building. A personal assistant that needs to recall your preferences is a files problem. A support agent that has to remember thousands of customer histories is a retrieval or memory-layer problem. Reach for the heaviest tool first and you'll spend a week building infrastructure you didn't need.

If you're starting from nothing, the first version is almost embarrassingly concrete, and you can have it running today. Give the agent one file describing you and how you like things done, and one for decisions already made. Have it read both at the start of every task and append to them at the end. When those files grow too big to load whole, that's your cue to move the older material into retrieval and pull back only what's relevant to the job. And once a week, read what the agent has written about you — that's how you catch a stale fact before it hardens into something the agent treats as true.

More memory isn't better memory

Before you rush to give your agent a perfect elephant's memory, though, there's a catch. A bigger memory isn't automatically a better one. The same lesson that applies inside a single context window applies to everything you feed back into it. A memory full of stale facts, half-finished decisions and things that were true last quarter doesn't make an agent wiser. It makes it confidently wrong.

Worse, memory compounds. A mistake that gets written into an agent's semantic memory doesn't just cause one bad answer; it gets recalled and reinforced until someone notices. Anything with a memory needs a way to forget, to correct, and to age things out. That's the part it's tempting to skip, and it's the part that separates a memory system that helps from one that slowly poisons itself.

None of this is exotic. It's mostly good hygiene — deciding what's worth remembering, keeping it clean, and loading it deliberately rather than hoarding everything on the off chance.

What separates a tool from a colleague

It's easy to treat memory as a technical detail, a feature you'll get round to. It's closer to the foundation everything else rests on. Memory is the line between a chatbot and a colleague. A tool answers the question in front of it and forgets you the moment you look away. A colleague remembers what you told them yesterday, learns how you work, and gets more useful the longer you work together. Everything we mean when we talk about AI agents becoming genuinely valuable — compounding, improving, earning trust — depends on their being able to remember.

The businesses that solve this first won't have the smartest models. Everyone has access to roughly the same models. They'll have agents that accumulate knowledge instead of resetting to zero every morning, and that difference gets wider every week it's left to compound.

If your AI agent can't remember what happened last Tuesday, it's a tool. When it can, it becomes infrastructure.