SimpleMem: Efficient Lifelong Memory for LLM Agents

A primary bottleneck in long-term interaction is context inflation, the accumulation of raw, low-entropy dialogue. For example, a large portion of interaction segments in the real-world consists of phatic chit-chat or redundant confirmations, which contribute little to downstream reasoning but consume substantial context capacity. To address this, we introduce a mechanism to actively filter and restructure information at the source.

First, incoming dialogue is segmented into overlapping sliding windows $W_{t}$ of fixed length, where each window represents a short contiguous span of recent interaction. These windows serve as the basic units for evaluating whether new information should be stored. Then we employ a non-linear gating mechanism, $\Phi_{gate}$, to evaluate the information density of these dialogue windows to determine which windows is used fo indexing. For each window $W_{t}$, we compute an information score $H(W_{T})$ that jointly captures the introduction of new entities and semantic novelty relative to the immediate interaction history $H_{\text{prev}}$.

Formally, let $\mathcal{E}_{new}$ denote the set of named entities that appear in $W_{t}$ but not in $H_{\text{prev}}$. The information score is defined as:

where $E(\cdot)$ denotes a semantic embedding function and $\alpha$ controls the relative importance of entity-level novelty and semantic divergence.

Windows whose information score falls below threshold $\tau_{\text{redundant}}$ are treated as redundant and excluded from memory construction, meaning that the window is neither stored nor further processed, preventing low-utility interaction content from entering the memory buffer. For informative windows, the system proceeds to a segmentation step:

For windows that pass the filter, we apply a segmentation function $\mathcal{F}_{\theta}$ to decompose each informative window into a set of context-independent memory units ${m_{k}}$. This transformation resolves dependencies implicit in conversational flow by converting entangled dialogue into self-contained factual or event-level statements. Formally, $\mathcal{F}{\theta}$ is composed of a coreference resolution module ($\Phi_{\text{coref}}$) and a temporal anchoring module ($\Phi_{\text{time}}$):

Here, $\Phi_{\text{extract}}$ identifies candidate factual statements, ($\Phi_{coref}$) replaces ambiguous pronouns with specific entity names (e.g., changing "He agreed" to "Bob agreed"), and $\Phi_{\text{time}}$ converts relative temporal expressions (e.g., transforming "next Friday" to "2025-10-24") into absolute ISO-8601 timestamps. This normalization ensures that each memory unit remains interpretable and valid independent of its original conversational context.

Then, the system need organize the resulting memory units to support efficient long-term storage and scalable retrieval. This stage consists of two components: (i) structured multi-view indexing for immediate access, and (ii) recursive consolidation for reducing redundancy and maintaining a compact memory topology over time.

To support flexible and precise retrieval, each memory unit is indexed through three complementary representations. First, at sematic layer, we map the entry to a dense vector space $\mathbf{v}_{k}$ using embedding models, which captures abstract meaning and enables fuzzy matching (e.g., retrieving "latte" when querying "hot drink"). Second, the Lexical Layer generates a sparse representation focusing on exact keyword matches and proper nouns, ensuring that specific entities are not diluted in vector space. Third, the Symbolic Layer extracts structured metadata, such as timestamps and entity types, to enable deterministic filtering logic. Formally, these projections form the comprehensive memory bank $\mathbb{M}$:

It allows the system to flexibly query information based on conceptual similarity, exact keyword matches, or structured metadata constraints.

While multi-view indexing supports efficient access, naively accumulating memory units over long interaction horizons leads to redundancy and fragmentation. To address this issue, we then introduces an asynchronous background consolidation process that incrementally reorganizes the memory topology. The consolidation mechanism identifies related memory units based on both semantic similarity and temporal proximity. For two memory units $m_{i}$ and $m_{j}$, we define an affinity score $\omega_{ij}$ as:

where the first term captures semantic relatedness and the second term biases the model toward grouping events with strong temporal proximity.

When a group of memory units forms a dense cluster $\mathcal{C}$, determined by pairwise affinities exceeding a threshold $\tau_{\text{cluster}}$, the system performs a consolidation step:

This operation synthesizes repetitive or closely related memory units into a higher-level abstract representation $M_{\text{abs}}$, which captures their shared semantic structure. For example, instead of maintaining numerous individual records such as ‘‘the user ordered a latte at 8:00 AM,’’ the system consolidates them into a single abstract pattern, e.g., ‘‘the user regularly drinks coffee in the morning.’’ The original fine-grained entries are archived, reducing the active memory size while preserving the ability to recover detailed information if needed. As a result, the active memory index remains compact, and retrieval complexity scales gracefully with long-term interaction history.

After memory entries are organized, another challenge to retrieve relevant information efficiently under constrained context budgets. Standard retrieval approaches typically fetch a fixed number of context entries, which often results in either insufficient information or token wastage. To address this, we introduces an adaptive query-aware retrieval mechanism that dynamically adjusts retrieval scope based on estimated query complexity, thereby improving retrieval efficiency without sacrificing reasoning accuracy.

First, we propose a hybrid scoring function for information retrieval, $\mathcal{S}(q,m_{k})$, which aggregates signals from the tri-layer index established in the second stage. For a given query $q$, the relevance score is computed as:

where the first term measures semantic similarity in the dense embedding space, the second term captures exact lexical relevance, and the indicator function $\mathbb{I}(\cdot)$ enforces hard symbolic constraints such as entity-based filters.

Then, based on the hybrid scoring, we can rank the candidate memories by relevance. However, retrieving a fixed number of top-ranked entries remains inefficient when query demands vary. To address this, we estimate the query complexity $C_{q}\in[0,1]$, which reflects whether a query can be resolved via direct fact lookup or requires multi-step reasoning over multiple memory entries. A lightweight classifier predicts $C_{q}$ based on query features such as length, syntactic structure, and abstraction level.

Based on this dynamic depth, the system modulates the retrieval scope. For low-complexity queries ($C_{q}\to 0$), the system retrieves only the top-$k_{min}$ high-level abstract memory entries or metadata summaries, minimizing token usage. Conversely, for high-complexity queries ($C_{q}\to 1$), it expands the scope to top-$k_{max}$, including a larger set of relevant entries, along with associated fine-grained details. The final context $\mathcal{C}_{final}$ is synthesized by concatenating these pruned results, ensuring high accuracy with minimal computational waste:

We evaluate SimpleMem across a diverse set of LLMs, ranging from high-capability proprietary models (GPT-4o series) to efficient open-source models (Qwen series). Tables 1 and 2 present the detailed performance comparison on the LoCoMo benchmark.

We conduct a comprehensive evaluation of computational efficiency, examining both end-to-end system latency and the scalability of memory indexing and retrieval. To assess practical deployment viability, we measured the full lifecycle costs on the LoCoMo-10 dataset using GPT-4.1-mini.

As illustrated in Table 3, SimpleMem exhibits superior efficiency across all operational phases. In terms of memory construction, our system achieves the fastest processing speed at 92.6 seconds per sample. This represents a dramatic improvement over existing baselines, outperforming Mem0 by approximately 14$\times$ (1350.9s) and A-Mem by over 50$\times$ (5140.5s). This massive speedup is directly attributable to our Semantic Structured Compression pipeline, which processes data in a streamlined single pass, thereby avoiding the complex graph updates required by Mem0 or the iterative summarization overheads inherent to A-Mem.

Beyond construction, SimpleMem also maintains the lowest retrieval latency at 388.3 seconds per sample, which is approximately 33% faster than LightMem and Mem0. This gain arises from the adaptive retrieval mechanism, which dynamically limits retrieval scope and prioritizes high-level abstract representations before accessing fine-grained details. By restricting retrieval to only the most relevant memory entries, the system avoids the expensive neighbor traversal and expansion operations that commonly dominate the latency of graph-based memory systems.

When considering the total time-to-insight, SimpleMem achieves a 4$\times$ speedup over Mem0 and a 12$\times$ speedup over A-Mem. Crucially, this efficiency does not come at the expense of performance. On the contrary, SimpleMem achieves the highest Average F1 among all compared methods. These results support our central claim that structured semantic compression and adaptive retrieval produce a more compact and effective reasoning substrate than raw context retention or graph-centric memory designs, enabling a superior balance between accuracy and computational efficiency.

To verify the claims that specific cognitive mechanisms correspond to computational gains, we conducted a component-wise ablation study using the GPT-4.1-mini backend. We investigate the contribution of three key components: (1) Semantic Structured Compression , (2) Recursive Consolidation, and (3) Adaptive Query-Aware Retrieval. The results are summarized in Table 4.

To illustrate how SimpleMem handles long-horizon conversational history, Figure 3 presents a representative multi-session example spanning two weeks and approximately 24,000 raw tokens. SimpleMem filters low-information dialogue during ingestion and retains only high-utility memory entries, reducing the stored memory to about 800 tokens without losing task-relevant content.