C An\sindex\sthat\ssurvives\sa\scommit
D 2026-09-14T16:50:47.236
E 2026-09-14T16:50:47.236 3f58202e745aaa7453cb5fa090f31d624a24dbb8
N text/x-markdown
U dmestas
W 3971
# An index that survives a commit
0.3.14 gave `zit log CHECKIN PATH` an index. One file's history on a mirror
of SQLite went from over a minute to under two seconds, and the release
notes had a table to prove it.
The table was true and the index was nearly useless.
## What was missing
The index is a fact the derive already computes and throws away: for each
check-in, the set of paths whose content moved against the primary parent.
Inverted by path, that is a file's history. It is stored as a `.zit-cache`
member, keyed — like the cluster and disposition maps — to the log tail and
the shun fingerprint, so a store that has moved on is never answered from a
stale map.
Keyed to the tail is the problem. A key that says *this index describes a
log ending at T* has exactly one useful answer to give: yes, or no. Commit
once and the answer is no, and the only thing the code could do with no was
throw the file away and walk all 36,702 check-ins again.
So the index helped precisely when nothing was happening. On a repository
anyone was actually working in — which is the only kind worth indexing —
the first PATH query after every commit paid the full rebuild. The
benchmark measured a case the benchmark had created.
## What changed
The stored index now records one more number: `n_rids`, how many log
entries it was built from. Validation moved from an exact tail match to a
prefix match, which is the whole idea in one line — an index built over a
shorter log is still a correct index *over that prefix*. It does not have
to be discarded. It has to be extended.
A query now adopts what is on disk, walks only the entries past `n_rids`,
folds the two together, and stores the union. Three cases, one path
through the code:
| on disk | walked |
|---|---|
| nothing | every check-in |
| an index over the whole log | none |
| an index over the log minus your last five commits | five |
On the SQLite mirror (131,558 artifacts), `log --long -n 5 README.md`:
```
no index 36,702 check-ins walked 55.4s
index, unchanged 0 check-ins walked 0.52s
a different path 0 check-ins walked 0.51s
```
The third line is the one that matters more than it looks. The index is
per-repository, not per-path: asking about a second file costs nothing,
because the work was never about the file you asked for.
## The test that would have caught it
There wasn't one, and that is the honest lesson here. Every 0.3.14 test
asserted the answer — byte-identical output against reconstruction, for
every tracked path. All of them passed. None of them asserted the *cost*,
so the version that rebuilt from scratch on every commit looked exactly
like the version that didn't.
`ZIT_FILEINDEX_WALK=1` now prints how many check-ins the walk covered, and
an end-to-end test commits into a repository with a warm index and asserts
that number is exactly the number of check-ins added. A regression to
rebuild-everything fails the gate instead of showing up as a slow
afternoon six weeks later.
## What this is not
The index narrows the walk. It does not decide the answer. `captureFileFacts`
diffs at F-card level, so a delta manifest restating a path with the hash it
already had reads as a change there while the path's effective content did
not move — an over-report, never an omission. Every candidate the index
names is still confirmed by the change predicate, which is what keeps the
answer exact rather than merely fast. `ZIT_NO_FILE_INDEX=1` answers by
reconstruction alone, and the test comparing the two for every tracked path
is unchanged.
The cache stays advisory: delete it and you pay one rebuild. A v1 file from
0.3.14 is ignored rather than misread.
## What is still slow
About half a second of every figure above is opening the repository. With
the walk at zero that is now the entire cost of a warm PATH query, and it is
paid again by every single CLI invocation. It is the next thing, and it is
tracked on its own.
Z cea3f97608e6e62441fe5ca157801f65