Agentic Systems: Memory Is a System, Not a Prompt Trick

This is Part 1 of Agentic Systems: Operating at Scale, the third series in a pillar on building agentic systems.

• Part 1: Memory Is a System, Not a Prompt Trick: the four stores that get collapsed into one word, and what each is for.
• Part 2: What Fan-Out Costs You: the invoice the architecture diagram does not show.
• Part 3: Containing a Bad Specialist: drift, correlated failure, and what containment actually requires.

The first two series covered how decisions get made and what surrounds them. This one is about running the result: where state lives, what it costs per decision, and how the system behaves when part of it goes quietly wrong. Starting with state, because the other two both depend on getting it right.

Table of Contents

• Why "More Context" Is Not an Architecture
• Four Stores, Four Lifetimes
• Why Derived State Has to Exist Separately
• Retrieval Is Not Memory
• The Failure Mode Is Always Staleness
• Specialists Should Not Share Working Context
• The Honest Bill
• When One Store Is Enough
• Conclusion

Why "More Context" Is Not an Architecture

The triage system needs to know that this service had a similar incident three weeks ago, that the team changed its on-call rotation, and that the last two pages for this alert class were false positives.

The default answer is to put all of it in the prompt. It works, for a while, and it fails in four ways that no amount of additional context length resolves.

Cost scales with what you include, on every call. Context is priced per token, and a panel of nine specialists multiplies whatever you include by nine, on every decision. History stuffed into context is not paid for once; it is paid for on every call that carries it, forever.

Relevance decays as the window grows. Model attention over long contexts is not uniform, and a fact placed among fifty thousand tokens of history is not reliably used just because it is present. "It is in the context" and "the model used it" are different claims, and the gap widens with length.

Nothing survives the session. Context is assembled, used, and discarded. If the only place a fact lives is a prompt, it exists until that call returns and then does not.

It cannot be queried. The calibration loop needs to aggregate across thousands of past decisions. Nothing about a prompt-shaped history supports that.

Each of these is a symptom of one underlying mistake: using a single mechanism for four different jobs that happen to share the word memory.

Four Stores, Four Lifetimes

Separating them makes each one tractable.

Working context is the only one that belongs in the prompt, and the discipline is to keep it minimal - what this specialist needs for this decision, not everything known about the situation. The other three are databases, and treating them as prompt content is the category error that produces all four failure modes above.

A signal-driven trading system separates the middle two explicitly: every signal, decision, trade, and outcome is persisted as the durable record, and a calibration job aggregates that history on a schedule into accuracy figures the orchestrator reads on every decision. The ledger is where truth lives. The calibration output is a small, fast, derived view of a judgement computed from that truth.

Why Derived State Has to Exist Separately

It is reasonable to ask why the orchestrator does not just query the ledger directly. It is the source of truth, after all, and a derived store is a cache with all the usual problems.

Three reasons, and they compound.

Cost grows with history. The orchestrator needs each specialist's calibrated accuracy on every decision. Computing that from raw outcomes means an aggregation over the full window, per specialist, per decision - work that grows as the ledger grows, on the critical path of every call.

The answer can move underneath a single decision. Aggregating live means outcomes landing mid-decision can change the weights between the moment specialist A is weighted and the moment specialist B is, producing an aggregate computed against two different versions of the world. A snapshot recomputed on a schedule is consistent in a way a live query is not, and consistency matters more here than freshness.

Calibration logic is expensive to trust. Bucketing, confidence stratification, outcome-lag handling - this is real logic, and running it on the decision path means every decision depends on it completing correctly and quickly. Running it on a schedule means a failure delays the update of a weight rather than blocking a decision.

The trade is explicit: derived state is stale by up to one recompute interval. That is almost always the right trade, and the interval is a real parameter - it sets the floor on how fast the system can respond to a specialist degrading, which is the subject of Part 3.

Retrieval Is Not Memory

The fourth store gets conflated with the others most often, usually because both get described as "giving the model knowledge."

A retrieval corpus - documents, runbooks, prior incidents, indexed for semantic search - answers what is relevant to this situation from a large body of reference material. The ledger answers what did this system do and how did it turn out. These are different questions over different data with different write patterns, and merging them produces a store that serves neither well.

A common and defensible deployment uses Postgres with pgvector for retrieval alongside the same database serving transactional and ledger needs - one store, on the reasoning that fewer moving parts wins until scale forces a split. That is a deployment decision, not a conceptual one, and the distinction is worth holding: sharing infrastructure between retrieval and ledger is often correct, while treating them as the same store conceptually is not. The failure that follows from conflating them is subtle - retrieved reference material starts being treated as evidence about the system's own behaviour, which it is not.

A useful test: if the answer should change because the system made a decision, it belongs in the ledger or derived state. If it should change because someone published a new document, it belongs in retrieval.

The Failure Mode Is Always Staleness

Three of the four stores can go stale, and each does so differently.

Derived state goes stale predictably, bounded by the recompute interval. This is the benign case: known bound, known trade.

The retrieval corpus goes stale invisibly, which is the dangerous one. A runbook that described the right procedure eighteen months ago still retrieves perfectly, ranks highly, and reads authoritatively. Nothing about the retrieval mechanism signals that the content is wrong, and the specialist consuming it has no way to tell. Retrieval needs document-level freshness metadata surfaced to whatever consumes it - and a specialist that cites a two-year-old runbook should be able to weight that differently from a current one.

Working context goes stale within a single decision if it was assembled early and the situation moved. Usually negligible; occasionally decisive for long-running decisions, where the right fix is to re-read rather than to carry the original snapshot forward.

The ledger, by construction, cannot go stale. It is append-only and describes the past, which is the property that makes it the thing everything else is derived from.

Specialists Should Not Share Working Context

One consequence follows directly from the panel's independence assumption and is easy to violate while trying to be efficient.

It is tempting to assemble one rich context and hand the same object to all nine specialists - it is simpler, and it looks like an optimisation. It quietly destroys the independence the panel's aggregation depends on. Specialists reading identical context have correlated inputs, so they have correlated errors: a misleading fact in the shared context misleads all of them in the same direction, and the orchestrator reads their agreement as corroboration at exactly the moment they are jointly wrong.

Each specialist should receive only what its own competency requires. The severity specialist does not need the service ownership map. Withholding it is not an oversight; it is what keeps that specialist's judgement genuinely independent of the ownership specialist's, and therefore what makes their agreement mean something when it occurs.

The Honest Bill

The silent-failure line is the operational one to watch. If the recompute job stops, nothing breaks - the orchestrator keeps reading the last good weights and keeps making decisions with them. The system continues to look healthy while its notion of which specialists to trust slowly detaches from reality. Derived state needs a freshness check that alerts when it has not been recomputed, because its failure mode is silence rather than error.

When One Store Is Enough

• No calibration, no derived state. If nothing weighs specialists by track record, there is nothing to derive, and the ledger alone is sufficient.
• No reference corpus. Plenty of agentic systems reason entirely over data passed in at decision time. Adding retrieval because it is conventional is cost with no consumer.
• Single-agent systems. With no panel, there is no independence to protect, and shared context is not a hazard.
• Volume too low for aggregation. A handful of decisions a day does not need a scheduled recompute; query the ledger directly and accept the cost.

Conclusion

Memory in an agentic system is four things wearing one word: working context assembled per call and discarded, an append-only ledger of what happened, derived state recomputed on a schedule for cheap consistent reads, and a retrieval corpus of reference material. Collapsing them into the prompt produces cost that scales with history, relevance that decays, state that does not survive the session, and a record that cannot be queried. Keeping them separate makes each tractable, and keeps working context scoped per specialist so the panel's independence survives.

Scoping context per specialist has an immediate consequence for what a decision costs, because nine specialists with nine contexts is nine calls. Part 2 works through that invoice.