itsbrill.ai

Test results

I came up with both of these and built them with my agents. Every figure is measured, and the method sits under each one.

agent-memory

My agents’ memory of past sessions, found by keyword search with no AI model.

Finding the right past conversation, against a popular open-source memory system

95.8%in the top 5

Within a point of its 96.6%, with no AI model at all

LongMemEval, a widely used public test for AI memory. 500 questions, scored with the other system’s own published method, so it’s like for like. It measures finding the right conversation, not answer quality. The other system’s figures are from its published run. Measured 30 October 2026.

Found in the top 5

agent-memory, no AI model95.8%
Popular open-source system96.6%

Found in the top 10

agent-memory, no AI model97.0%
Popular open-source system98.2%
The other system has about 60,000 GitHub stars and 146 contributors, and searches by meaning with an AI model.

What it takes to run

0AI models

Plain keyword search, about 4,900 lines of Python and no outside libraries

No AI model and no vector database. agent-memory gets within a point of the other system with plain keyword search, and runs all 500 questions in about 2 seconds on a Mac Studio, building its index included. I came up with the design and the rules, and my agents built much of it with me.

agent-memoryPopular open-source system
How it searchesBy keyword, with SQLite’s full-text searchBy meaning, with a vector database
AI modelNoneAn embedding model of 30 to 300 MB
Tools for agents1147
Built byMe and my agents, 45 commits since July 2026146 contributors, about 2,500 commits since April 2026

How agent-memory works

agent-si

Facts only, each quoted word for word from a source.

Time to search agent-si for a fact, with no model involved

13ms

Median of 250 searches on my own working brain, 808 records and 3,509 sources. agent-si 1.11.0, 29 October 2026.

  1. Read one record1.7 ms
  2. Look up a fact2.0 ms
  3. Filter records2.0 ms
  4. Search13.3 ms
  5. Build the startup brief24.1 ms
Median time for each kind of call, in milliseconds. The engine’s own time, not counting the agent or its model.

What an agent reads to start a session, agent-si against plain markdown notes

11×less to read

10,895 bytes against 117,185 bytes

Measured on a 40-person test company. Less to read means fewer tokens, so lower cost.

agent-si10,895 bytes

plain markdown notes117,185 bytes

Each block is 10,895 bytes, the size of agent-si’s startup.

Flat as it grows

At four times the history, the markdown notes grow to 477,986 bytes. agent-si’s startup and every answer stay the same size.

At four times the history

agent-si10,895 bytes

plain markdown notes477,986 bytes

Each block is 10,895 bytes, as above.

Answers that repeated a planted misleading value, agent-si against plain markdown notes

0of 72

against 12 of 72

24 questions about the same test company, each run 3 times, with the same model and tool for both.

Same test, with strict grading. An answer fails if it mentions any planted misleading value.

agent-si0 of 72

plain markdown notes12 of 72

Each square is one answer.

Same answer every time

Reads are byte-for-byte identical, tested on every release (1,769 tests).

The same read three times, and the same bytes each time.

How I tested it

How I work

The proof. The systems I run my agents on, and what surprised me along the way.

Work with me

Let’s get you and your team working with agents the way I do.

Work with me
Enter to openEsc to close