Artifact Description [59e75a6fef]

Artifact 59e75a6fefe9fdb951bc6bd1e3479b2fa557f80eabb65e8ea73d060d76ce2092:

• File src/cmd/verify.zig
  • 2026-07-31 21:12:22 — part of check-in [b224ffeb40] on branch logstore-slice2 — logstore: openReadOnly scans header-only; verified_on_open is gone

    The speedup reaches production. openReadOnly is the path every CLI verb takes -- Repository.open falls back to it, and Repository.logStore, the content channel behind cat, artifact, checkout and sync, calls it with no cache in front. It scanned every payload byte on every invocation:

    10k frames 42 MB 42.2ms -> 10.3ms 10k frames 410 MB 298.0ms -> 12.3ms 100k frames 416 MB 511.9ms -> 115.4ms 100 frames 419 MB 321.4ms -> 0.10ms

    Open cost is now a function of frame count, not repository size.

    verified_on_open was the flag openReadOnly set to tell readFrameAt that the scan had already checked every frame, so reads could skip their own crc. With no scan doing that, the exemption had nothing left to rest on and the flag is deleted. readFrameAt now crc-checks unconditionally: it is the only thing standing between a rotted payload and a caller that believes it, and restoring any skip there means restoring the scan.

    Detection is relocated, not removed. A payload bitflip no longer voids the open -- the rest of the repository stays usable -- and surfaces the first time anyone reads those bytes: "zit cat" and "zit artifact" both refuse with 'repository is corrupt', and "zit verify" names the artifact and the offset. That verb shipped first for exactly this reason.

    openForVerify is deleted. It existed because openReadOnly both aborted on a damaged store and granted read-time trust, neither of which verify could accept; both properties are gone, it became byte-identical to openReadOnly, and its own doc comment said to remove it in this commit.

    The DST prove-it-can-fail arm is inverted rather than deleted. It asserted that openReadOnly WOULD serve latent rot silently, across a seed range where at least one corpus lands a payload rot on a raw frame whose header survived. That hazard is gone, so the same range now asserts the opposite: the read path never serves rot as success. trust_on_read goes with it -- there are no longer two paths to select between. Losing the eager whole-log scan cost nothing here, because per-read verification is what was catching this all along.

    Net -40 lines.

    Verified: fmt clean, test 2327/2327, gate, e2e 671/671, dst 6 seeds x 1000 steps. The Slice 1 tripwire -- the assertion that openReadOnly still rejected eagerly, written to fail if the flag was deleted without an fsck verb -- fired on this commit, which is the one it was aimed at. (user: dmestas, size: 17838) [annotate] [blame] [check-ins using]

  • 2026-07-31 21:15:04 — part of check-in [6d9172d488] on branch trunk — merge logstore-slice2: the open-cost work reaches production

    openReadOnly is what every CLI verb opens through, and until now it still checksummed every payload byte on every invocation -- so nothing users ran had gotten faster. It scans header-only now:

    10k frames 42 MB 42.2ms -> 10.3ms 10k frames 410 MB 298.0ms -> 12.3ms 100k frames 416 MB 511.9ms -> 115.4ms 100 frames 419 MB 321.4ms -> 0.10ms

    Open cost is a function of frame count rather than repository size. On a store of a few large artifacts it is effectively free.

    This completes the sequence: vendor a faster crc, measure the path production actually takes, scan header-only on the append path, ship the fsck verb, then delete the eager check. Each step was gated on the one before it, and the deletion was deliberately last because it is the only one that gives something up.

    What is given up: opening a repository no longer detects payload corruption. It surfaces on the read instead -- readFrameAt crc-checks every frame unconditionally now -- or on demand through "zit verify". That is the split git draws between ordinary operation and fsck, and it was signed off knowing the tradeoff. A damaged store also stays usable for everything that does not touch the damaged bytes, which the old eager check made impossible.

    Slices 0 and 4 are closed too: the 4 KiB chunk widening landed inside verify, the only place that loop still runs, and the sidecar index is dead on its own gate -- the 100k-frame shape was to justify it by exceeding 150ms and came in at 119ms. mmap is dead alongside it, separately, because a mapping would keep shunned bytes resident and readable until every mapping dropped, which defeats reclamation rather than merely slowing it. Neither rejection is recorded at the logstore extension comment yet; that is the one piece of follow-up this leaves behind.

    Verified on the merged tree: fmt clean, test 2327/2327, gate 19/19. On the branch tip, additionally e2e 671/671 and dst 6 seeds x 1000 steps. (user: dmestas, size: 17838) [annotate] [blame] [check-ins using]

  • 2026-08-01 07:22:43 — part of check-in [584314dcb7] on branch sub-manifest — merge trunk: land the logstore slices, repair the sweep's rot oracle

    Trunk's Slice 1/2 made both open paths header-only, so a rotted payload now opens clean and surfaces at content() instead of at open. The storage sweep still assumed open was the tripwire, so two of its own claims were wrong once the merge combined them with this branch's store:

    - crashCheckOnce repaired only "if the open itself reports corruption", a precondition Slice 2 removed. Injected rot reached the invariant check and read as data loss. NeedsRepair, private to the harness, now carries "refused AND genuinely damaged" to the repair that already existed.

    - rotWithinFrame spelled the per-frame overhead as 22, the header alone, so a frame's own trailing frame_crc32 counted as inert pad. A bitflip in those four bytes makes a reader recompute a correct checksum, compare it to a corrupted stored one, and refuse the frame; the sweep called that data loss. logstore now exports frame_fixed_overhead and the sweep derives it instead of respelling it.

    Both are trunk defects, not merge damage: trunk alone fails dst at seed 8. The merge only shifted the draws far enough to reach the first one at seed 3, inside the default gate's range.

    cli.zig's verb enum was the single textual conflict: a pure union of trunk's verify and this branch's submanifest and import-git.

    fmt, test, gate, dst green. (user: dmestas, size: 17838) [annotate] [blame] [check-ins using]