Station 6 · Agent Memory

How an agent remembers

An agent's context window is a small, disappearing whiteboard — it fills up and wipes clean every session. Memory is how the agent keeps hold of what it did five steps ago, and who you were last week.

6 of 10 · ~7 min

What you'll walk away with

Why an agent needs memory — and what breaks without it.
Short-term vs long-term memory — inside the prompt vs stored outside.
Episodic vs semantic — remembering events vs remembering facts.
The practical toolkit: RAG, user profiles, summarization, and forgetting.

Concept 1

Why an agent needs memory

The LLM only "sees" what's inside its context window — the block of text (the prompt) it reads before answering. That window is limited and it resets. Nothing is remembered automatically.

The window is finite. A long conversation eventually overflows — the oldest turns must fall off the edge to make room.
The window resets. Close the chat and open a new one, and the agent starts blank. It doesn't know your name, your last project, or what it just decided.
An agent loops for many steps. Without a way to carry results forward, it re-does work, contradicts itself, or forgets the goal halfway through.

🧠 Analogy: working memory vs. a notebook

Your working memory holds a phone number just long enough to dial it — then it's gone. That's the context window. A notebook is where you write things down so you can flip back to them days later. An agent needs both: the fast whiteboard and the notebook it can reach for when the whiteboard runs out of space.

Concept 2

Short-term vs. long-term memory

The cleanest way to split agent memory is by where it lives: inside the current prompt, or stored somewhere outside it.

In the window

Short-term memory

The current conversation — everything sitting in the context window right now. Instant to use, but limited in size and gone when the session ends.

Outside the window

Long-term memory

Saved outside the model — a vector database, a SQL table, or a custom store. Nearly unlimited, survives across sessions, and is pulled back into the window only when relevant.

The key move is retrieval: long-term memory doesn't live in the prompt, so the agent must search its store, find the relevant piece, and paste it into the context window before answering. That "search and paste it back in" step is exactly the RAG pattern you met with embeddings in Station 2.

Concept 3

Episodic vs. semantic memory

Long-term memory comes in two flavors, borrowed from how psychologists describe human memory.

🎬 Episodic memory

Specific events that happened. "Last Tuesday you asked me to book a flight to Denver." Time-stamped, personal, story-shaped.

📚 Semantic memory

General facts the agent has learned or been told. "You're vegetarian." "Your company uses Python." No date attached — just true things.

🔑 The core idea

Memory turns a stateless model into something that feels like it knows you. Episodic memory lets it say "last time we tried X…"; semantic memory lets it skip questions it already has the answer to. Both are just text the agent retrieves and drops back into the context window.

Concept 4

Maintaining memory — the practical bit

A store isn't magic. Real agents lean on four techniques to keep memory useful without blowing past the context limit.

RAG & vector databases — turn each memory into an embedding (a vector), and at query time retrieve the closest matches to what the user just said. Only the relevant slivers come back into the prompt.
User-profile storage — a small, structured record of durable facts (name, preferences, goals) that gets pasted in every session, so the agent never has to re-learn the basics.
Summarization / compression — when the conversation gets long, replace the oldest turns with a short summary. Ten messages become two sentences that still fit the window.
Forgetting / aging — deliberately drop stale or low-value memories (an old address, a one-off question) so the store stays small and retrieval stays sharp.

Concept 5

Store → retrieve, in slow motion

Here's the whole loop as pseudocode. Watch the token spans on the right — that's how much of the limited context window each step consumes.

memory.py — store then retrieve

# --- SESSION 1: the agent learns something and stores it ---
remember("User is vegetarian and lives in Austin.")
  # → embed to a vector, save in the store   [~12 tokens, lives OUTSIDE the window]

# ...days later, brand-new session, context window is empty again...

# --- SESSION 2: user asks something related ---
user = "Find me a good dinner spot for tonight."       [~9 tokens]

Thought   → "What do I already know about this user?"
Action    → recall(user)                              # search the store
Observe   → "User is vegetarian and lives in Austin."  [~12 tokens pulled BACK in]

Answer    → "Here are 3 vegetarian spots in Austin..."
  # prompt sent to the model = system + user + recalled memory
  # total ≈ 33 tokens — tiny, because we retrieved only what mattered

why this worksThe store holds thousands of facts, but only the one relevant line re-enters the window. That's RAG: search outside, paste the best match back in, keep the prompt small.

🗂️ Analogy: a librarian, not a backpack

A bad agent carries every past conversation in its backpack (the prompt) until it can't lift it. A good agent works like a librarian: the archive is huge and off to the side, and when you ask a question they walk over, pull the one book you need, and bring it to the desk. Retrieval beats hoarding.

Checkpoint

Your agent needs to recall a user preference from a chat three weeks ago. Where does that live, and how does it get used?

It's still in the context window — the model never forgets. In long-term storage (e.g. a vector DB); the agent retrieves it and pastes it back into the window. It's gone forever — agents can't remember past a single session.

The context window reset weeks ago, so the preference can't still be "in" it. It only survives if it was saved to long-term memory outside the model. To use it, the agent searches that store (RAG), finds the match, and drops it back into the current prompt — the store-then-retrieve loop.

See it yourself — 5 minutes

Catch an agent remembering (and forgetting)

Open an AI tool that has a memory feature (ChatGPT's "Memory", or Claude with projects/instructions) and probe it:

Tell it a durable fact: "Remember that I'm learning about AI agents." Watch it save that to a profile / long-term store — not just this chat.
Start a fresh conversation and ask "What am I studying?" If it answers, it retrieved from long-term memory. If it's blank, that fact only lived in short-term (context) memory.
Ask it to "summarize our conversation so far." That's compression in action — shrinking a long history so it still fits the window.

Recap. The context window is short-term memory — fast, small, and wiped every session. Long-term memory lives outside the model in a vector DB, SQL, or profile store, split into episodic (events) and semantic (facts). Agents keep it usable with RAG retrieval, user profiles, summarization, and forgetting — always the same move: store outside, retrieve the relevant bit, paste it back into the window. Next, we zoom out to how these pieces get wired together.

← BackStation 5 · Prompting Your Agent Next →Station 7 · Agent Architectures