Timeline

openReadOnly is what every CLI verb opens through, and until now it still checksummed every payload byte on every invocation -- so nothing users ran had gotten faster. It scans header-only now:

10k frames 42 MB 42.2ms -> 10.3ms 10k frames 410 MB 298.0ms -> 12.3ms 100k frames 416 MB 511.9ms -> 115.4ms 100 frames 419 MB 321.4ms -> 0.10ms

Open cost is a function of frame count rather than repository size. On a store of a few large artifacts it is effectively free.

This completes the sequence: vendor a faster crc, measure the path production actually takes, scan header-only on the append path, ship the fsck verb, then delete the eager check. Each step was gated on the one before it, and the deletion was deliberately last because it is the only one that gives something up.

What is given up: opening a repository no longer detects payload corruption. It surfaces on the read instead -- readFrameAt crc-checks every frame unconditionally now -- or on demand through "zit verify". That is the split git draws between ordinary operation and fsck, and it was signed off knowing the tradeoff. A damaged store also stays usable for everything that does not touch the damaged bytes, which the old eager check made impossible.

Slices 0 and 4 are closed too: the 4 KiB chunk widening landed inside verify, the only place that loop still runs, and the sidecar index is dead on its own gate -- the 100k-frame shape was to justify it by exceeding 150ms and came in at 119ms. mmap is dead alongside it, separately, because a mapping would keep shunned bytes resident and readable until every mapping dropped, which defeats reclamation rather than merely slowing it. Neither rejection is recorded at the logstore extension comment yet; that is the one piece of follow-up this leaves behind.

Verified on the merged tree: fmt clean, test 2327/2327, gate 19/19. On the branch tip, additionally e2e 671/671 and dst 6 seeds x 1000 steps. check-in: 6d9172d488 user: dmestas tags: trunk

The speedup reaches production. openReadOnly is the path every CLI verb takes -- Repository.open falls back to it, and Repository.logStore, the content channel behind cat, artifact, checkout and sync, calls it with no cache in front. It scanned every payload byte on every invocation:

10k frames 42 MB 42.2ms -> 10.3ms 10k frames 410 MB 298.0ms -> 12.3ms 100k frames 416 MB 511.9ms -> 115.4ms 100 frames 419 MB 321.4ms -> 0.10ms

Open cost is now a function of frame count, not repository size.

verified_on_open was the flag openReadOnly set to tell readFrameAt that the scan had already checked every frame, so reads could skip their own crc. With no scan doing that, the exemption had nothing left to rest on and the flag is deleted. readFrameAt now crc-checks unconditionally: it is the only thing standing between a rotted payload and a caller that believes it, and restoring any skip there means restoring the scan.

Detection is relocated, not removed. A payload bitflip no longer voids the open -- the rest of the repository stays usable -- and surfaces the first time anyone reads those bytes: "zit cat" and "zit artifact" both refuse with 'repository is corrupt', and "zit verify" names the artifact and the offset. That verb shipped first for exactly this reason.

openForVerify is deleted. It existed because openReadOnly both aborted on a damaged store and granted read-time trust, neither of which verify could accept; both properties are gone, it became byte-identical to openReadOnly, and its own doc comment said to remove it in this commit.

The DST prove-it-can-fail arm is inverted rather than deleted. It asserted that openReadOnly WOULD serve latent rot silently, across a seed range where at least one corpus lands a payload rot on a raw frame whose header survived. That hazard is gone, so the same range now asserts the opposite: the read path never serves rot as success. trust_on_read goes with it -- there are no longer two paths to select between. Losing the eager whole-log scan cost nothing here, because per-read verification is what was catching this all along.

Net -40 lines.

Verified: fmt clean, test 2327/2327, gate, e2e 671/671, dst 6 seeds x 1000 steps. The Slice 1 tripwire -- the assertion that openReadOnly still rejected eagerly, written to fail if the flag was deleted without an fsck verb -- fired on this commit, which is the one it was aimed at. closed check-in: b224ffeb40 user: dmestas tags: logstore-slice2

Slice 2 will extend header-only scanning to openReadOnly and delete verified_on_open, which removes eager corruption detection entirely. This is the on-demand replacement that has to exist first -- the same split git makes between normal operation and git fsck.

Two passes over the store. FrameVerifier walks every frame under .verify_payloads, which is the mode the open paths are shedding; it is built on StorageFile rather than an opened LogStore, so it still works on a store that will not open -- the case an fsck exists for. Then openForVerify opens .headers_only and reconstructs each artifact, which catches a frame whose checksum is honest about bytes that are not what their name claims.

Both passes report rather than abort. A payload CRC fault carries a header-proven frame_len, so the walk steps over it and inventories the rest; only a header-window fault, whose length cannot be trusted, stops the walk. A torn tail is not a fault -- open recovers it silently, and it means an interrupted write, not rot. An unreadable payload is a finding too, not the end of the inventory.

Exit codes stay 0/1/2 as every other verb documents them; 2 remains usage error and never corruption. Findings stream to stderr as they are found, the clean summary goes to stdout, and on any fault stdout is empty.

Scope stops at content artifacts. A manifest whose bytes hash correctly is byte-identical to what commit wrote and what sync validated on ingest, so re-parsing it would catch a zit bug rather than storage rot. The help text says so.

openForVerify is deliberately not openReadOnly: openReadOnly sets verified_on_open, which makes readFrameAt skip the per-read whole-frame CRC, and it scans .verify_payloads, so on exactly the damaged store an operator is fsck-ing it aborts before pass 2 runs. Slice 2 makes the two identical and openForVerify is deleted in that commit; its doc comment says so.

The chunk buffer in scanOneFrame's payload loop widens from 4 KiB to 64 KiB, matching cloneInto's existing precedent. Post-Slice-2 that loop runs only under verify, where an operator's wall-clock is what it costs.

zit verify is not an RFC-mandated verb. repo-state 7.3 blesses full rebuild as a fallback, but the verb's shape is a local design decision, not a protocol requirement.

Verified: fmt clean, test 2327/2327, gate 19/19, e2e 671/671. Four mutants, all compiling, each red for its own semantic reason: pointing the frame pass at .headers_only, stubbing artifact.verify to true, dropping the unreadable-store tally, and reverting the payload read to a hard error. Exit arms smoke-checked against the built binary. closed check-in: bcb9f8eb6e user: dmestas tags: logstore-open-cost

zig build test was 9m13s, and the tests it actually ran were a small fraction of that: it also compiled the e2e and smoke harnesses in the pinned ReleaseSafe and then never executed them. test is now unit tests only -- 2m54s cold, 1m42s warm -- and the new gate step carries those two compile proofs.

The dst, bench and smoke UNIT tests deliberately stay on test. They are unit tests despite their siblings' names, zig build dst/smoke run the simulation and the interop check rather than them, and CI invokes the lanes one by one and never invokes gate -- so moving them would have dropped them from CI while every step still looked green. ci_test's declared_steps records gate as intentionally not-in-CI, and names that failure mode for the next lane someone adds. check-in: 7363238ad5 user: dmestas tags: trunk