Case studySystems2026
Zero-token recall: building memory outside the model
The bottleneck was never the model. It was starting from zero every session.
- 221durable facts in memory, measured August 2026
- ~5msto answer a recall question, before any model runs
- 0tokens spent on the deterministic recall path
- Context
- I run doctoral research, a full-time role, and a publishing pipeline through AI every day. At that intensity the bottleneck is context: every session that re-explains history pays rent on knowledge I already own.
- What I built
- A local, file-based second brain whose deterministic index answers recall questions in about five milliseconds at zero tokens before any model runs, with hybrid semantic search as the fallback, automated session capture, and a consolidation pass at the end of each working day. Around it, a fleet of scheduled agents on always-on infrastructure, wired to real tools over MCP.
- Headline result
- Recall that costs 0 tokens and answers in about five milliseconds, and memory that compounds instead of resetting.
Specification
- Role
- Designer and operator
- Timeline
- 2026, running daily
- Stack
- Plain markdown plus a deterministic index, local hybrid search (keyword, vectors, rerank), scheduled agents on an always-on host, MCP integrations
- Status
- In daily use, and this site itself runs on the same discipline.
The problem
Working through AI at high volume has two failure modes. The first is context loss: every new session rediscovers what previous sessions already knew, at full price. The second is the reflex that fixes it by pushing more context into every prompt, so cost scales with effort instead of with value.
I wanted the opposite property. Knowledge should be written once and recalled at near zero cost forever, and the expensive model should only ever see the one piece of context that answers. The system also had to survive vendor and model changes, run on my own hardware, and maintain itself, because a memory system that depends on discipline stops being maintained the first busy week.
The approach
Four decisions, in the order they matter.
Plain files over a database
Facts are atomic markdown files with structured frontmatter, catalogued in a single index. Files are versionable, greppable, readable without tooling, and model-agnostic, so the system outlives any vendor choice.
Deterministic code before the model
Most recall questions do not need a model. A scoring pass over one master index finds the right file and the right section in about five milliseconds at zero token cost, and the model reads only that section. I chose an index-first ladder over stuffing context into every prompt because context behaves like attention: the less of it spent remembering, the more goes to the problem.
Hybrid semantic search is the fallback, not the default. Paraphrase and needle-in-haystack questions fall through to local hybrid search, keyword plus vectors plus rerank, running entirely on my machine. It triggers only when the cheap ladder scores low, so the expensive path stays the exception.
A fleet of scheduled agents with narrow mandates
Cron-driven agents run on always-on infrastructure: monitoring, research, first drafts. Credentials are pooled and managed centrally, agents self-report status, and failures surface in a morning brief. Agents draft. Nothing ships without my review.
Real tools over MCP. The system connects to email, calendar, meeting transcripts, image and video generation, and SEO and analytics tooling through Model Context Protocol, so agents act on real systems instead of producing text about them.
Memory maintained by process, not discipline
Every local AI working session is captured and digested automatically on a schedule. A consolidation pass at the end of each working day promotes durable facts into the canonical store and reconciles drift. Nothing depends on remembering to write things down.
- 01
Question
A recall question arrives from any session on the machine.
- 02
Deterministic index
One scoring pass over the master index names the file and the section. About five milliseconds, zero tokens.
- 03
Hybrid search
Only questions the cheap pass scores low on fall through to local keyword, vector and rerank search.
- 04
One section
The model reads the section that answers, and nothing else.
Context behaves like attention. The less of it spent remembering, the more goes to the problem.
The outcome
The measured result is a recall path that answers in about five milliseconds and spends 0 tokens before any model runs. The structural result is memory that compounds: any session, on any surface of my machine, starts from the same facts instead of rediscovering them.
The same discipline runs through my published research, which benchmarks models from open weight to frontier and selects them by task complexity and cost. And it runs through this site: the answering questions here is built on one curated corpus with grounded answers only, the same pattern at public scale.
What I would do differently
Move the consolidation pass onto always-on hardware sooner. It currently runs at the end of each working day because my main machine sleeps. And log recall misses from the first week, because the misses are the map of what the index lacks.
This system is personal infrastructure, so nothing here is abstracted. I am glad to walk through it live.