Artifact [4b9f9d0c68]

Artifact 4b9f9d0c68023aa126c953d0d6f73936eb122cca7d914c5e472932c4de07d558:

• File docs/RELEASE-NOTES-0.3.18.md — part of check-in [09d191e634] at 2026-09-16 22:11:26 on branch trunk — Release 0.3.18

  The file-history index is built by the derive at ingest, fossil's way. (user: dmestas, size: 5012) [annotate] [blame] [check-ins using]

# zit 0.3.18

zit 0.3.18 builds the file-history index the way fossil builds `mlink`: at
ingest, as each check-in is derived, rather than on demand by the first
query. A file's history is an index lookup from a repository's first query
onward, the whole-repository cost lands where clone and import already pay,
and the build can no longer fail to converge on a restarting forge.

No artifact, tag namespace, sync message, or on-disk repository format changed.
The index format moved to version 4; a version-3 index is rebuilt once.

## The index is built by the derive

fossil answers `finfo` from a table it fills as each check-in artifact arrives:
one row per changed file, indexed by name, so the query is a lookup. zit's
derive — the pass that turns the log into leaves, tags and events, run once
when a store is first opened and extended over the tail on every append — was
already computing the identical fact for each check-in and writing it into the
checkpoint's record stream. It just never kept it as an index. The index was
instead rebuilt from scratch by a separate full pass on the first `log PATH`.

The derive now hands each check-in's change set, inverted by path, to an index
builder as it links the check-in, and stores the result beside the checkpoint.
Trees are diffed by S card there like everywhere else. On a fresh clone or
import the index exists before the first query; on append, the tail-extend
path adopts the stored index and extends it over the new check-ins; a cache
hit computes nothing, so the zero-parse open is untouched.

| pkgsrc submanifest mirror, 218,444 check-ins, fresh clone to first file-history answer | 0.3.17 | 0.3.18 |
|---|---|---|
| wall clock | 11.7 min (derive, then the index pass) | **7.1 min** (derive with the index inline) + 1.15s |
| peak memory | 1.21 GB | 1.61 GB |
| where the index cost lands | first query, in one non-resumable pass | clone, checkpointed with the derive |
| every later query | ~1s | ~1s |

The derive alone was about five minutes; building the index inside it adds
about two, because both share one read of every manifest, and retires the
separate 6.8-minute pass. The price is 0.4 GB of peak memory for the index
builder during that one derive.

## The on-query build is now the fallback, and it resumes

A store whose cache predates this release still builds its index on the first
`log PATH`. That build now checkpoints every 8,192 entries, so a build a
restart interrupts resumes from the last chunk instead of from zero. A forge
that restarts more often than the whole build takes now converges across
restarts; before, every restart spent the full duration and left nothing.

Measured on the pkgsrc mirror carrying a 0.3.17 index: the new binary refused
that index and began the rebuild; killed at 90 seconds, it had saved 909,312
of 1,249,347 entries, marked incomplete; the next run walked only the
remaining check-ins, 87 seconds, and finished with a complete index; the query
after that walked nothing and answered in 1.06s.

A partial index is recorded as partial: version 4 stores how many entries
existed when it was written, an index is complete only when it covers all of
them, and the derive's tail-extend path adopts only a complete one. Without
that, a partial index whose key matched the checkpoint's could be adopted as
whole and claim coverage over the gap — an omission from file history, found
in review before it shipped.

One limit of the fallback, stated rather than buried: it covers by entry
count, so a phantom filled after a build is never revisited by a later resume.
The derive walks by frame offset and re-covers it, which is one more reason
the derive-built index is primary.

## `zit crosslink` reports a tree check-in's files

The crosslink stream used to say `files unknown` for a tree check-in, because
reporting them meant flattening the tree and the derive runs on every cold
open. With the S-card diff a tree costs about what changed, so the stream now
carries its files like any other check-in's. `files unknown` still means what
it always meant — a parent the store does not hold, or a sub-manifest that is
absent or unreadable — and never a tree as such.

## Compatibility

- No on-disk repository or wire format changed; Fossil RFC revision 00 remains
  the target.
- `.zit-cache/finfo` is version 4. A version-3 index is ignored and rebuilt
  once, by the derive on the next cache build or by the resumable fallback on
  the next `log PATH`. The cache stays advisory; deleting it is always safe.
- No command or flag changed. `ZIT_FILEINDEX_WALK` remains a diagnostic seam.

## Qualification

`zig build test`, `e2e`, `smoke` (Fossil interop against a real `fossil`),
`muscle` (41 differential steps), `git-import-e2e`, `git-export-e2e`, `dst`,
and `cross`, plus the three-runner CI battery. The pkgsrc figures are release
builds against the submanifest mirror in the benchmark corpus with the cache
removed before each cold run; both columns ran alone on the same machine.