Long Context, Better Agent Memory

Part of a series on running AI workflows across model providers — this one is about a question we get a lot: "if the context window is huge, do I even need to think about memory?"

Every current Claude model — Opus 5, Sonnet 5, Fable 5.1, Mythos 5.1 — ships with a 1M-token context window. That's a real, significant number, big enough to fit a decent-sized codebase or a long stack of documents in a single call. It's also led to a specific bad habit we keep seeing: teams treating the context window as a replacement for memory design, instead of one tool among a few. This post is about the difference, and about why Anthropic itself, of all companies, just shipped evidence that the difference matters.

The tempting shortcut

The appeal is obvious: if the window is big enough, why build anything? Just paste in the whole conversation, every document, every tool result, on every call. It works, technically, right up until it doesn't:

Cost compounds. Every prior turn you re-send gets re-billed. A conversation that's cheap at turn three can get expensive by turn thirty, because you're paying for the same early context over and over.

Latency creeps up. Bigger inputs take longer to process, even on a model built to handle them. What felt instant early in a session can feel sluggish an hour in.

Precision degrades. Models are good at finding relevant information in a large context, not perfect at it. A detail buried in message four of a forty-message history is more likely to get missed than one that was recently and explicitly surfaced.

There's still a ceiling. A million tokens is enormous, but it's not infinite, and a system designed around "just keep appending" eventually hits it — usually at the worst possible moment, mid-conversation, with no graceful fallback built in.

The evidence that even Anthropic agrees

On September 14, 2026, Anthropic shipped on-demand compaction as a beta API feature — the ability to compact a conversation on request, getting back a signed summary block you can send in place of the full message history on later calls, while still preserving recent turns word-for-word alongside it.

Sit with what that means for a second. The company that makes the model with the biggest context window in its own lineup just built a dedicated tool for not using all of it. That's not a contradiction — it's an acknowledgment that a bigger window changes how painful naive context management is, not whether context management is necessary. If Anthropic is investing in compaction, "just rely on the window" was never the actual endpoint.

Three strategies, not one

┌─────────────────────┐  ┌─────────────────────┐  ┌─────────────────────┐
│ Stuff full context  │  │ Session memory      │  │ Knowledge base RAG  │
│ (simple, costly)    │  │ (sessionId)         │  │ (corpus too large)  │
└─────────────────────┘  └─────────────────────┘  └─────────────────────┘

A bigger window makes the left column less painful. It doesn't make it the right default.

Stuffing everything into context is simple to build and expensive to run at scale. It has a place — short-lived interactions, prototypes, cases where the total content genuinely is small — but it's a starting point, not an architecture.

Structured session memory — what DNotifier ties to a sessionId — is built for anything shaped like an ongoing conversation. Instead of your application code manually reassembling a message array from scratch on every call, the platform tracks history per session and assembles the right context automatically:

const sessionId = "onboarding-user-4471";

await notifier.sendAI({
  senderId: userId,
  sessionId,
  message: { text: "What's the next step in setting up my account?" },
  saveHistory: true,
});

// Minutes or messages later, same session — prior context is already there
await notifier.sendAI({
  senderId: userId,
  sessionId,
  message: { text: "I finished that, what's next?" },
  saveHistory: true,
});

This keeps cost proportional to what's actually relevant, keeps latency roughly flat as a conversation grows, and keeps recent turns explicit instead of hoping the model finds them in a growing pile.

Knowledge base retrieval — DNotifier's built-in RAG layer — solves a different problem entirely: not "what did we just talk about," but "what does this document set say," where the document set is too large to fit in any context window at all, however generous. Semantic search pulls the handful of relevant passages out of a corpus of any size, without you provisioning or tuning a vector database separately.

The strategies aren't competing with each other. A well-built support agent, for instance, typically uses session memory for the conversation it's currently having and knowledge base retrieval for the product documentation it's drawing answers from — two different memory problems, two different tools, both running underneath the same sendAI() call.

So when does the big context window actually matter?

It's genuinely valuable for a narrower set of jobs than "everything." Long-document analysis where you want the model reasoning over an entire contract or codebase at once, in a single pass, is a real use case a 1M-token window makes newly practical. So is a case where you deliberately want the model to have broad context for one specific, high-value call — not as a substitute for ongoing memory management, but as a tool you reach for on purpose, for a specific task, rather than a default you never think about again.

The mistake isn't using a large context window. It's treating its size as a reason to stop designing memory deliberately.

A rule of thumb worth keeping

If what you're building has the shape of a conversation — turns, back-and-forth, a relationship that persists across a session — reach for structured session memory first. If what you're building needs to draw on a body of content too large to reasonably fit anywhere — documentation, policies, a knowledge corpus — reach for retrieval. Reach for a large single context window when a task genuinely calls for the model to reason over a large amount of material in one deliberate pass, not as a way to avoid deciding between the first two.

Frequently asked questions

Doesn't a 1M-token window make RAG obsolete?

No — even at 1M tokens, most real knowledge bases (documentation sites, policy libraries, historical records) are larger than that, and retrieval is also just cheaper and faster for the common case of "answer this one question" versus "load everything, every time."

Should I use on-demand compaction instead of DNotifier's session memory?

They solve adjacent but different problems. Anthropic's compaction is a way to shrink an already-large conversation before resending it to Claude specifically. DNotifier's session memory is the layer deciding what gets sent in the first place, provider-agnostically. Using DNotifier's session model well often means you need compaction less urgently, since you're not defaulting to full-history resends to begin with.

How do I know if I'm hitting the "precision degrades" problem?

A practical test: ask about something stated early in a long session and see if the answer is as sharp as it would be for something stated in the last two turns. If there's a noticeable gap, that's the buried-detail problem showing up in your own product, not just a theoretical concern.

Does this change if I'm using a smaller, cheaper Claude model instead of Opus 5?

The underlying tradeoffs are the same regardless of which Claude model is answering — all four current models share the 1M-token ceiling. Memory strategy is a property of your application's design, not of which specific model you've routed a given call to.

Is there a cost difference between session memory and stuffing context manually?

Usually, yes, meaningfully — session memory only carries forward what's relevant to the current exchange rather than the entire history by default, which tends to keep per-call token counts (and therefore cost) lower as a conversation grows longer.

The Bottom Line

That's the full series, at least for now: getting Claude connected, understanding DNotifier's MCP support, weighing Anthropic's own Managed Agents platform, building a real multi-agent pipeline, and now the memory question underneath all of it. The pattern that holds across every post here is the same one: a good model is necessary and not sufficient — the infrastructure around it is where production systems actually get built or quietly fall apart.

Read part one: Claude on DNotifier: Setup Guide. Read part four: Building Research Agents with Claude. Explore dnotifier.com.