MemoryOS: Memory OS of AI Agent

📝 Paper Summary

Memory organization Memory recall

MemoryOS provides AI agents with a comprehensive operating system-inspired memory framework, utilizing hierarchical storage (short, mid, long-term) and heat-based paging to maintain long-term coherence and personalization.

Core Problem

LLMs rely on fixed-length context windows, which fail to sustain coherence across long temporal gaps, resulting in fragmented memory, factual inconsistencies, and a lack of personalized continuity.

Why it matters:

Current memory solutions operate in isolation (focusing only on storage or retrieval), lacking a unified system for comprehensive management.
Long-term coherence is critical for applications requiring persistent user adaptation and stable persona representation over extended interactions.

Concrete Example: In a long conversation where a user mentions a preference early on, a standard LLM with a fixed window might forget this detail after many turns, leading to a later response that contradicts the user's previously stated preference.

Key Novelty

OS-Inspired Hierarchical Memory Management

Structures memory like an operating system using a three-tier hierarchy: Short-Term (STM), Mid-Term (MTM), and Long-Term Personal Memory (LPM).
Implements a 'Segmented Paging' strategy where dialogue pages are grouped by topic segments, managed via heat-based eviction (frequency, engagement, recency) rather than simple FIFO.

Evaluation Highlights

+49.11% average improvement on F1 score compared to baselines on the LoCoMo benchmark using GPT-4o-mini.
+46.18% average improvement on BLEU-1 score compared to baselines on the LoCoMo benchmark using GPT-4o-mini.

Breakthrough Assessment

7/10

Strong empirical gains on LoCoMo and a cohesive system design inspired by OS principles. While hierarchical memory is not new, the specific integration of segmented paging and heat-based eviction offers a solid engineering advance.

⚙️ Technical Details

Problem Definition

Setting: Long-context conversational interaction requiring maintenance of user and agent personas over time.

Inputs: Current user query Q, model response R, and timestamp T, forming a dialogue page.

Outputs: Contextually coherent and personalized response generated by retrieving from hierarchical memory.

Pipeline Flow

Input Processing: User Query -> Memory Retrieval (STM, MTM, LPM)
Generation: Retrieved Context -> Response Generation
Maintenance: New Interaction -> STM Update -> (Overflow) -> MTM Update -> (Heat Threshold) -> LPM Update

System Modules

Memory Storage

Organizes data into STM (pages), MTM (segments of pages), and LPM (profiles/traits)

Model or implementation: Hierarchical structure (STM/MTM/LPM)

Memory Updating

Manages data migration: STM->MTM via FIFO; MTM->LPM via Segmented Paging and Heat scores

Model or implementation: Algorithmic logic (Heat equation)

Memory Retrieval

Retrieves relevant context using 2-stage retrieval (Segment then Page) for MTM, and semantic search for LPM

Model or implementation: Dense retrieval (Semantic similarity)

Response Generation

Integrates retrieved memories into a final prompt to generate the answer

Model or implementation: LLM (e.g., GPT-4o-mini)

Novel Architectural Elements

Segmented Paging storage architecture mimicking OS memory management for Mid-Term Memory
Heat-based eviction and promotion mechanism (Heat Score equation) governing data flow between MTM and LPM

Modeling

Base Model: GPT-4o-mini (used for experiments)

Compute: Not reported in the paper (Inference/System paper)

Comparison to Prior Work

vs. MemGPT: MemoryOS uses implicit heat-based promotion/eviction rather than explicit LLM-generated function calls for memory management.
vs. MemoryBank: MemoryOS structures memory into segments and pages (topics) rather than just flat vector entries.
vs. Generative Agents [not cited in paper]: MemoryOS focuses on conversational utility and explicit persona management rather than simulating independent agent behavior in a sandbox.

Limitations

Relies on LLM calls for summarizing segments and pages, which may incur latency and cost.
The efficacy of the heat score relies on manually tuned coefficients (alpha, beta, gamma).
Experiments primarily conducted with GPT-4o-mini; performance on smaller open-source models is less emphasized.

Reproducibility

Code: https://github.com/BAI-LAB/MemoryOS

Code is publicly available at https://github.com/BAI-LAB/MemoryOS. The paper relies on the LoCoMo benchmark. Specific hyperparameters like heat coefficients are provided (e.g., threshold tau=5, time constant mu=1e+7).

📊 Experiments & Results

Evaluation Setup

Long-term conversational evaluation measuring coherence and personalization.

Benchmarks:

LoCoMo (Long Context Memory Benchmark)

Metrics:

F1 score
BLEU-1
ROUGE-L
Distinct-1
Statistical methodology: Not explicitly reported in the paper

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
LoCoMo	F1	Not reported in the paper	Not reported in the paper	Not reported in the paper
LoCoMo	BLEU-1	Not reported in the paper	Not reported in the paper	Not reported in the paper

Main Takeaways

MemoryOS demonstrates significant relative improvements (approx. 50%) over baselines on LoCoMo, indicating superior handling of long-context constraints.
The hierarchical structure effectively balances recent context (STM) with topic-based retrieval (MTM) and persistent personalization (LPM).
The segmented paging and heat-based update mechanisms successfully automate memory maintenance without requiring manual user intervention.

📚 Prerequisite Knowledge

Prerequisites

Understanding of LLM context window limitations
Basic operating system memory concepts (paging, segmentation, eviction)
Vector databases and semantic retrieval

Key Terms

STM: Short-Term Memory—stores recent dialogue pages in a fixed-length FIFO queue for immediate context.

MTM: Mid-Term Memory—stores recurring topic summaries organized into segments and pages, updated via heat scores.

LPM: Long-term Personal Memory—stores persistent user profiles, knowledge bases, and traits.

Dialogue Page: The basic unit of storage containing a query, response, timestamp, and meta-summary.

Segmented Paging: A storage strategy where pages are grouped into logical segments (topics), allowing fine-grained management.

Heat Score: A metric determining memory retention based on visit count, interaction length, and time decay.

LoCoMo: A benchmark dataset designed to evaluate long-term conversational memory capabilities.

FIFO: First-In-First-Out—a queue processing method where the oldest items are removed first.

Jacard similarity: A statistic used for gauging the similarity and diversity of sample sets (used here for keyword matching).