Agent Memory & Context Systems

Agent memory is not one feature. It is a stack of capture, compression, retrieval, ownership, evaluation, and workflow design choices that determine whether an agent becomes more useful over time or merely repeats itself with confidence.

Visual navigation Use the cluster tools to review this hub as a navigable system, not only as prose: - Agent Memory Cluster Dashboard - Agent Memory Cluster - Local visuals

Executive map Use this page as the hub for the memory cluster. Detailed memory pages remain in the flat wiki/ directory as supporting pages; this page carries the primary reading path.

!Illustrative agent memory context stack

*Generated conceptual illustration, not source evidence: raw history narrows into active context, durable memory, retrieval, and evaluation loops.*

flowchart TD
    A["Source history"] --> B{"Context policy"}
    B --> C["Compress state"]
    B --> D["Retrieve evidence"]
    C --> E["Derived memory"]
    D --> E
    E --> F["Architecture choice"]
    F --> G["Harness ownership"]
    G --> H["Workflow support"]
    H --> I{"Evaluate"}
    I --> J["Correct memory"]
    I --> K["Revise policy"]
    J --> E
    K --> B

Core thesis

The memory cluster should be read as one system:

• LLM Memory explains what memory has to store, retrieve, forget, and prove.
• Agent Memory Architectures explains the design space for storing and reusing memory across agent tasks.
• Context Compaction explains how active reasoning state is compressed when hot context becomes expensive.
• Memory Transfer Learning explains which forms of memory travel across domains and which create negative transfer.
• Open Harnesses explains why whoever owns memory, context policy, and workflow state owns much of the product.
• Second Brain Systems and Chief of Staff Agents show how the same architecture becomes useful in personal and executive work.

The deeper insight is that memory is not valuable because it is long. It is valuable when it preserves the right abstraction at the right time, with enough provenance to be corrected.

Two newer papers sharpen that point. Skill-RAG shows that failed retrieval can require typed repair, not just more retrieval. The world-knowledge paper shows that an agent can treat environment exploration as a preparation phase, producing a compact guidebook that functions like task-ready memory.

A newer personal-context MCP source adds the user-facing version of the same idea: personal context should be modular, portable, machine-readable, maintainable as files, and selectively routable to different agents rather than trapped inside one model provider's memory feature.

A newer agentic-memory source makes the hub more concrete by splitting memory into three jobs: continuity, task state, and learning. That framing is useful because it prevents memory design from collapsing into one storage question. User identity, current tool observations, and lessons from prior failures all need different write, retrieval, and forgetting policies.

Primary reading path

1. Start here for the memory stack and design questions.
2. Read LLM Memory when deciding what should persist, what should be forgotten, and how provenance should work.
3. Read Agent Memory Architectures when choosing between markdown memory, vector retrieval, graph-vector hybrids, replay, and consolidation.
4. Read Context Compaction when the main constraint is active-context cost or long-thread continuity.
5. Read Open Harnesses when the strategic question is ownership, portability, or lock-in.
6. Read Persistent Agent Threads when memory needs to support long-running work rather than isolated tasks.

The stack

Memory jobs

The agentic-memory source gives a compact design test:

These jobs can share infrastructure, but they should not share an undifferentiated write policy. A preference, a live blocker, and a postmortem lesson have different lifetimes and different proof requirements.

Design rule

Do not ask “does this agent have memory?” Ask:

1. What does it store?
2. Who owns it?
3. What gets compressed?
4. What gets retrieved?
5. What can be forgotten?
6. What evidence proves the memory improved the work?

Failure-aware retrieval

Memory and retrieval systems should not treat every miss the same way.

Skill-RAG’s useful contribution is a small retrieval repair vocabulary:

This matters for agent memory because retrieval failure is often an alignment problem between the task, the corpus, and the model’s intermediate state. Simply retrieving more can amplify drift.

World knowledge as environment memory

The world-knowledge paper adds a complementary memory artifact: a guidebook produced before a specific downstream task.

Useful world knowledge captures:

• environment structure
• important pages, paths, or affordances
• task-relevant facts and entities
• navigation constraints
• what has already been inspected
• where future retrieval should start

This is not the same as a raw transcript. It is a compiled, reusable map of an environment. In this vault’s terms, it sits between raw/ evidence and the active context used for future work.

Personal context as a portable portfolio

The personal context MCP source proposes a practical memory package:

The design principles are durable:

• markdown first
• modular, not one giant profile
• living, not static
• portable across agents and model providers
• exposable through files, GitHub, MCP, or other controlled interfaces
• maintainable by interview, revision, and recurring agent-assisted updates rather than one-time setup
• selectively routable so different agents receive only the context they actually need

This connects memory to Open Harnesses: the more valuable personal context becomes, the more important ownership and portability become.

Context readiness is a memory problem, not just a data problem

A useful enterprise lesson from the source is that "data readiness" often really means context readiness.

Organizations fail with agents when:

• documents exist but are not structured for machine use
• access boundaries block the right context from reaching the agent
• useful operating knowledge lives only in chat history or tribal memory
• teams ship generic copilots without exposing task-specific context
• provider-native memory creates convenience but weak portability

This matters because many deployment failures are not failures of model capability. They are failures of context packaging, access, and routing.

Implementation pattern

Use a three-part memory design before adding new memory surfaces:

Most memory failures come from mixing these layers. Working context becomes stale, durable memory becomes cluttered with trivia, or evidence becomes inaccessible when the system needs to explain itself.

Skill-RAG adds one more review question: when memory fails, did the system diagnose the kind of failure, or did it only repeat retrieval?

The personal-context portfolio adds another: can the user inspect, edit, export, and route the memory to a different agent without starting over?

MCP is a serving layer, not the memory itself

The source adds a practical implementation distinction that belongs in this hub.

An MCP server is best understood here as a controlled serving layer over memory artifacts, not as the memory artifact itself.

That means:

• the durable value lives in the files and their maintenance discipline
• MCP exposes those files through a standard protocol
• local versus remote deployment changes access patterns, auth, and risk
• troubleshooting is often about ports, filenames, auth models, and resource mapping, not about the concept itself

This is useful because it separates:

• memory design
• memory ownership
• memory transport
• runtime access

The stack becomes more legible when those are not collapsed into one idea.

Retrieval is most of the design

The agentic-memory source adds a blunt implementation warning: storage is the easy part; retrieval design carries most of the value. In practical agents, the memory loop is:

1. retrieve before the model call
2. reason and act with the retrieved context
3. write back after the action
4. periodically decay, score, consolidate, or delete memories

That loop makes memory stateful, but it also creates risk. If retrieval brings the wrong memories forward, the agent can become more confidently wrong than a stateless model. Memory systems need relevance, importance, recency, and provenance controls, not only a larger store.

Visual reading cues

Read the image and diagrams together:

Visual references worth preserving

The raw proactive-agent paper includes system diagrams that are useful for this cluster because they show memory, demand detection, and assistance as a closed loop:

!PASK proactive-agent architecture

Use source images sparingly. They should clarify architecture, workflow, or evidence. Decorative images do not improve the wiki.