Test notes

Five narrative pages before the scoreboard.

Each page explains what is being compared, how the plugin was configured, why some competitors are not run in their most expensive mode, and what the result can honestly claim.

75public artifacts

The full archive stays close to the narratives, so every claim can be inspected from a report or runner.

Control test

Sibyl Scale365: the control run before the fight starts

I would rather not open with a leaderboard. The honest starting point is the control run: a year of synthetic business life, 500 companies, 1,500 stakeholders, 209,000 write calls, and the one question I keep coming back to. Can an agent retrieve current operational state without dumping the whole past into the prompt?

Read the Sibyl baseline

HonchoA direct 42k-record comparison with the same 250-question suite, the same answering model and a clear caveat around Honcho async processing.Mem0The Mem0 test is not a dunk. It is a cost question: what happens when a memory plugin needs LLM extraction before it becomes competitive?HindsightHindsight finds meaningful memories in local mode, but the average context climbs so high that Sonnet becomes the expensive part of the test.MnemosyneMnemosyne is tested in FTS-only mode, so this page explains the result, the missing vector path and the rerun needed before any strong claim.

Evidence layer

The archive contains the reports, raw result indexes and rerun scripts.

Use the five pages for the readable story. Use the archive when you want the raw material: benchmark reports, security rounds, migration tests and runners.

Open the full archive