Comparison | Mem0
Sibyl vs Mem0: Scale365, 191k records and extraction cost
Why Mem0 is tested with infer=False, what that measures, and what enabling inference would cost.
Full test note
The Mem0 story is really about cost, not a dunk. The local baseline I tested is real and reproducible, and limited on purpose. A stronger Mem0 mode exists, but the reports put a price tag on it: extraction stops being a footnote once the corpus hits 41M tokens.
Read the full test pageVerdict
Mem0 retrieves 92/350 and reaches 105/350 after Sonnet, with 1,720 average context tokens and about 92 minutes of ingestion.
Mem0 is tested with local Qdrant, fastembed, BM25 sparse and `infer=False`. The test measures a local baseline without LLM extraction, not Mem0 in its smartest configuration.
Comparison table
| Metric | Sibyl | Mem0 |
|---|---|---|
| Retrieval | 350/350 | 92/350 |
| Sonnet answers | 344/350 | 105/350 |
| Average context | 228 tokens | 1,720 tokens |
| Ingestion | 48 s | 5,512 s |
| Estimated answer cost | $0.64 | $2.76 |
Methodological caveat
Mem0 is not tested with `infer=True`. That is an explicit choice: enabling LLM extraction on this corpus would introduce major cost and ingestion time into the benchmark.
Why not the best possible configuration?
- The report estimates roughly 41M minimum tokens to pass through LLM extraction for this corpus.
- Depending on the model, that means around 50 to 100 dollars with a cheaper model, or about 445 dollars with Sonnet.
- The expected ingestion time then becomes a central part of product cost, not a detail to remove from the table.
Rerun the test
python scripts/run_mem0_365d_500c_category_baseline.py