> ## Documentation Index
> Fetch the complete documentation index at: https://metacognition-fdc534de-master.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Benchmarks and methodology

> LoCoMo and LongMemEval_S scores, with category tables, latency, tokens, and methodology.

export const Bar = ({label, pct, value, win}) => <div style={{
  display: "flex",
  alignItems: "center",
  gap: "10px",
  margin: "6px 0",
  fontSize: "13px",
  lineHeight: 1.4
}}>
    <span style={{
  minWidth: "180px",
  maxWidth: "180px",
  fontWeight: win ? 600 : 400,
  opacity: win ? 1 : 0.78
}}>{label}</span>
    <div style={{
  flex: 1,
  height: "16px",
  background: "rgba(127,127,127,0.14)",
  borderRadius: "3px",
  overflow: "hidden"
}}>
      <div style={{
  width: `${Math.min(100, pct)}%`,
  height: "100%",
  background: win ? "#F32C05" : "rgba(127,127,127,0.55)",
  borderRadius: "3px"
}}></div>
    </div>
    <span style={{
  minWidth: "60px",
  textAlign: "right",
  fontWeight: win ? 600 : 400,
  opacity: win ? 1 : 0.78,
  fontVariantNumeric: "tabular-nums"
}}>{value}</span>
  </div>;

export const Chart = ({title, children}) => <div style={{
  margin: "20px 0",
  padding: "18px 18px 14px",
  border: "1px solid rgba(127,127,127,0.18)",
  borderRadius: "10px"
}}>
    <div style={{
  fontSize: "12px",
  opacity: 0.6,
  marginBottom: "10px",
  textTransform: "uppercase",
  letterSpacing: "0.05em",
  fontWeight: 600
}}>{title}</div>
    {children}
  </div>;

Tex scores **93.3%** overall on **LoCoMo** with the full system. Tex scores **92.2%** on **LongMemEval\_S** with active retrieval only.

This page shows the category tables, latency, tokens, and methodology behind those numbers.

<CardGroup cols={2}>
  <Card title="LoCoMo · 93.3%" icon="trophy">
    Full Tex system vs published baselines. EverMemOS was the prior headline number at **92.3%**.
  </Card>

  <Card title="LongMemEval_S · 92.2%" icon="trophy">
    Active retrieval track vs other retrieval-first systems. Emergence AI posted **86.0%** on comparable reporting.
  </Card>
</CardGroup>

<Info>
  We generated answers with **gpt-4o-mini** and graded them with **gpt-4o** in an LLM-as-judge setup. Each evaluation ran on a **single** machine. The exact setup is in **Methodology** below.
</Info>

## LoCoMo

LoCoMo evaluates long-conversation memory across 10 multi-session conversations with **1,984 questions** spanning 5 categories: single-hop retrieval, multi-hop reasoning, temporal reasoning, open-domain inference, and adversarial abstention.

### Per-category results

| Category    | **Tex Full** | Tex Active | MemMachine v0.2 | Hindsight (Gemini-3) |
| ----------- | ------------ | ---------- | --------------- | -------------------- |
| Single-Hop  | **96.08%**   | 88.23%     | 94.41%          | 86.17%               |
| Multi-Hop   | **92.14%**   | 78.21%     | 89.72%          | 70.83%               |
| Temporal    | **91.90%**   | 87.23%     | 89.10%          | 83.80%               |
| Open-Domain | 94.79%       | 67.71%     | 75.00%          | **95.12%**           |
| Adversarial | **99.33%**   | 97.31%     | —               | —                    |
| **Overall** | **93.3%**    | 87.7%      | 91.69%          | 89.6%                |

### Competitive landscape

<Chart title="LoCoMo · Overall Accuracy">
  <Bar label="Tex (Full System)" pct={93.3} value="93.3%" win />

  <Bar label="EverMemOS" pct={92.3} value="92.3%" />

  <Bar label="MemMachine v0.2" pct={91.7} value="91.7%" />

  <Bar label="Hindsight" pct={89.6} value="89.6%" />

  <Bar label="Tex (Active Only)" pct={87.7} value="87.7%" win />

  <Bar label="Zep" pct={85} value="~85%" />

  <Bar label="Letta / MemGPT" pct={83} value="~83%" />

  <Bar label="Supermemory" pct={70} value="~70%" />

  <Bar label="Mem0" pct={66} value="~66%" />
</Chart>

<Tip>
  On adversarial items Tex is **99.33%**. It declines to answer when the transcript does not support an answer. If a public benchmark skips that bucket, compare headline numbers carefully.
</Tip>

## LongMemEval\_S

LongMemEval\_S evaluates memory over **500 questions across \~48 sessions each (\~115K tokens)**, testing information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention.

### Per-ability results

| Ability           | **Tex Active** | Mastra OM (gpt-4o) | Emergence AI | Supermemory | Oracle GPT-4o | Zep   |
| ----------------- | -------------- | ------------------ | ------------ | ----------- | ------------- | ----- |
| Abstention        | **100%**       | —                  | —            | —           | —             | —     |
| Knowledge Updates | **95.8%**      | 85.9%              | 83.3%        | 88.5%       | 85.9%         | 83.3% |
| Info Extraction   | 93.3%          | 82.1%              | **100%**     | 97.1%       | **100%**      | 92.9% |
| Temporal          | **91.3%**      | 85.7%              | 85.7%        | 76.7%       | 76.7%         | 62.4% |
| Multi-Session     | **87.6%**      | 79.7%              | 81.2%        | 71.4%       | 80.5%         | 57.9% |
| **Overall**       | **92.2%**      | 84.2%              | 86.0%        | 81.6%       | 82.4%         | 71.2% |

### Competitive landscape

<Chart title="LongMemEval_S · Retrieval Systems">
  <Bar label="Tex (Active Only)" pct={92.2} value="92.2%" win />

  <Bar label="Emergence AI (Internal)" pct={86.0} value="86.0%" />

  <Bar label="Mastra OM (gpt-4o)" pct={84.2} value="84.2%" />

  <Bar label="Emergence AI (Simple)" pct={82.4} value="82.4%" />

  <Bar label="Supermemory" pct={81.6} value="81.6%" />

  <Bar label="Emergence AI (Fast)" pct={79.0} value="79.0%" />

  <Bar label="Zep" pct={71.2} value="71.2%" />

  <Bar label="Full Context GPT-4o" pct={63.8} value="63.8%" />

  <Bar label="Naive RAG" pct={52.0} value="52.0%" />
</Chart>

<Chart title="Oracle baselines (evidence provided up front — no retrieval)">
  <Bar label="Oracle GPT-o3" pct={92.0} value="92.0%" />

  <Bar label="Oracle Accumulator" pct={88.6} value="88.6%" />

  <Bar label="Oracle GPT-4o" pct={82.4} value="82.4%" />
</Chart>

<Tip>
  **92.2%** is close to Oracle GPT-o3 (**92.0%**) and well above Oracle GPT-4o (**82.4%**). That means retrieval is finding evidence close to what you would hand-pick for each question.
</Tip>

## Latency

| Configuration         | p50         | p90     | End-to-end (including the model that reads context) |
| --------------------- | ----------- | ------- | --------------------------------------------------- |
| **Tex (Active)**      | **\~120ms** | \~200ms | \~0.6s                                              |
| **Tex (Full System)** | \~350ms     | \~500ms | \~1.0s                                              |

<Chart title="Retrieval latency vs competitors · lower is better">
  <Bar label="Tex (Active Only)" pct={2.1} value="120ms" win />

  <Bar label="Tex (Full System)" pct={6.2} value="350ms" win />

  <Bar label="Supermemory" pct={5.3} value="300ms" />

  <Bar label="Emergence AI" pct={63.7} value="3.6s" />

  <Bar label="Zep" pct={100} value="5.65s" />
</Chart>

## Token efficiency

### Per-question breakdown (Tex)

| Component                                        | Tokens      |
| ------------------------------------------------ | ----------- |
| System prompt                                    | \~138       |
| User template                                    | \~28        |
| Retrieved context (avg 24 fragments × 44 tokens) | \~1,047     |
| Question                                         | \~20        |
| **Total input per question**                     | **\~1,233** |
| Output (answer)                                  | \~30        |

### LLM tokens consumed on LoCoMo (1,984 questions)

| System            | Ingestion LLM | Reader   | Total    | Accuracy  |
| ----------------- | ------------- | -------- | -------- | --------- |
| **Tex**           | **0**         | **2.4M** | **2.4M** | **93.3%** |
| MemMachine Memory | not disclosed | 4.2M     | 4.2M+    | 87.5%     |
| MemMachine Agent  | not disclosed | 8.6M     | 8.6M+    | 91.7%     |
| Mem0              | not disclosed | 19.2M    | 19.2M+   | \~66%     |

<Chart title="Tokens per correct answer · lower is better">
  <Bar label="Mem0" pct={100} value="~14,650+" />

  <Bar label="MemMachine (Agent)" pct={32.2} value="~4,720+" />

  <Bar label="MemMachine (Memory)" pct={16.5} value="~2,420+" />

  <Bar label="Tex (Full System)" pct={8.8} value="~1,296" win />
</Chart>

<Tip>
  Ingest here costs **zero LLM tokens**. This run used offline embeddings only. If a system runs an LLM per message during ingest, that cost should show up in token usage.
</Tip>

### Headline efficiency claims

Compared with published Mem0 numbers, Tex is about **27%** higher on accuracy, uses roughly **87%** fewer tokens, and runs at about **95%** lower latency. Against MemMachine's memory-only configuration, Tex is about **5.8%** more accurate on **43%** fewer tokens. On LoCoMo, Tex also has the lowest tokens-per-correct-answer in this comparison: about **1,296**.

## Ablation: what each part of the pipeline adds

On LongMemEval\_S, removing pieces of the retrieval pipeline changes accuracy like this:

| Capability                    | What it does                                      | Δ accuracy        |
| ----------------------------- | ------------------------------------------------- | ----------------- |
| Adaptive retrieval            | Adjusts retrieval depth + strategy per query type | +3.6%             |
| Query understanding           | LLM-free semantic expansion                       | +1.9%             |
| Multi-stage relevance scoring | Progressive filter + rerank                       | +2.4%             |
| Structured reasoning          | Enumerate / count across fragments                | +3.1%             |
| Temporal awareness            | Resolve relative dates, build chronology          | +3.0%             |
| Combined pipeline             | Everything working together                       | +1.6%             |
|                               | **Base → Full System**                            | **76.6% → 92.2%** |

## Methodology

Answers came from **`gpt-4o-mini`**. We graded them with an LLM-as-judge setup built on **`gpt-4o`**, using category-specific prompts, binary pass/fail per item, and a straight average inside each category.

**LoCoMo** used all ten provided conversations (**1,984** question-answer pairs) against the **full Tex system** (not retrieval-only). **LongMemEval\_S** used **500** questions with about **48** sessions each (\~115K tokens per trace) against **Tex Active** retrieval only.

The full pipeline ran on a **single** machine. There was no multi-node orchestration.

## On our roadmap for evals

We plan to add **MemoryAgentBench (ICLR 2026)**, stronger multi-step reasoning for questions that need counting or arithmetic over evidence, and **LongMemEval\_M** for questions that span hundreds of sessions.

## References

1. Maharana et al. *LoCoMo: Long-Context Conversations for Evaluating Conversational Memory.* EMNLP 2024.
2. Wu et al. *LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory.* ICLR 2025.
3. Hu et al. *Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions.* ICLR 2026.
4. MemMachine. *MemMachine v0.2 Delivers Top Scores on LoCoMo.* Dec 2025.
5. Emergence AI. *SOTA on LongMemEval with RAG.* 2025.
6. Supermemory. *State-of-the-Art Agent Memory Research.* 2026.
7. Mastra. *Observational Memory: 95% on LongMemEval.* 2026.

<Card title="Run the quickstart" icon="play" href="/quickstart" horizontal>
  Install the SDK, store one turn, and print a recall result.
</Card>