Timeline

Opening a repository checksummed every payload byte, so open cost scaled with the whole store rather than with what the caller asked for. The append path now scans header-only:

10k frames 42 MB 42.2ms -> 11.2ms 10k frames 410 MB 298.0ms -> 12.7ms 100k frames 416 MB 511.9ms -> 119.2ms 100 frames 419 MB 321.4ms -> 0.10ms

openReadOnly -- the path every CLI verb reaches -- deliberately still verifies payloads. Nothing has moved on the production path yet. That is Slice 2, and it is gated on this merge's other half: zit verify, the on-demand fsck that has to exist before eager detection is removed.

The 100k-frame run settles a question the plan left open. Slice 4's gate was 'build the sidecar index only if that shape exceeds 150ms after the scan change'; it lands at 119ms. The sidecar is dead, and mmap with it -- separately, a mapping would keep shunned bytes resident and readable until every mapping dropped, which defeats the reclamation guarantee rather than merely slowing it. Both should be recorded as evaluated-and-rejected at the logstore extension comment; that is not done here.

Also adds --mode append|read-only to the open benchmark. Every open number this effort had produced measured LogStore.open, which no CLI verb enters; the two modes agreed within noise before the scan change, which is what makes the after numbers above a comparison rather than two guesses.

Verified on the merged tree: fmt clean, test 2327/2327. gate 19/19 and e2e 671/671 were watched green at the branch tip, which is content-identical -- trunk did not move between the branch point and this merge. check-in: 90bd1b8015 user: dmestas tags: trunk

Slice 2 will extend header-only scanning to openReadOnly and delete verified_on_open, which removes eager corruption detection entirely. This is the on-demand replacement that has to exist first -- the same split git makes between normal operation and git fsck.

Two passes over the store. FrameVerifier walks every frame under .verify_payloads, which is the mode the open paths are shedding; it is built on StorageFile rather than an opened LogStore, so it still works on a store that will not open -- the case an fsck exists for. Then openForVerify opens .headers_only and reconstructs each artifact, which catches a frame whose checksum is honest about bytes that are not what their name claims.

Both passes report rather than abort. A payload CRC fault carries a header-proven frame_len, so the walk steps over it and inventories the rest; only a header-window fault, whose length cannot be trusted, stops the walk. A torn tail is not a fault -- open recovers it silently, and it means an interrupted write, not rot. An unreadable payload is a finding too, not the end of the inventory.

Exit codes stay 0/1/2 as every other verb documents them; 2 remains usage error and never corruption. Findings stream to stderr as they are found, the clean summary goes to stdout, and on any fault stdout is empty.

Scope stops at content artifacts. A manifest whose bytes hash correctly is byte-identical to what commit wrote and what sync validated on ingest, so re-parsing it would catch a zit bug rather than storage rot. The help text says so.

openForVerify is deliberately not openReadOnly: openReadOnly sets verified_on_open, which makes readFrameAt skip the per-read whole-frame CRC, and it scans .verify_payloads, so on exactly the damaged store an operator is fsck-ing it aborts before pass 2 runs. Slice 2 makes the two identical and openForVerify is deleted in that commit; its doc comment says so.

The chunk buffer in scanOneFrame's payload loop widens from 4 KiB to 64 KiB, matching cloneInto's existing precedent. Post-Slice-2 that loop runs only under verify, where an operator's wall-clock is what it costs.

zit verify is not an RFC-mandated verb. repo-state 7.3 blesses full rebuild as a fallback, but the verb's shape is a local design decision, not a protocol requirement.

Verified: fmt clean, test 2327/2327, gate 19/19, e2e 671/671. Four mutants, all compiling, each red for its own semantic reason: pointing the frame pass at .headers_only, stubbing artifact.verify to true, dropping the unreadable-store tally, and reverting the payload read to a hard error. Exit arms smoke-checked against the built binary. closed check-in: bcb9f8eb6e user: dmestas tags: logstore-open-cost

The open-time scan streamed every payload byte through the crc, making open cost O(repository bytes) rather than O(what the caller asked for). ScanMode.headers_only validates each frame's header window and then steps over the payload by frameSpan arithmetic, leaving open cost O(frame count).

Measured, same binary, both modes in one run:

10k frames 42 MB 42.2ms -> 11.2ms 10k frames 410 MB 298.0ms -> 12.7ms 100k frames 416 MB 511.9ms -> 119.2ms 100 frames 419 MB 321.4ms -> 0.10ms

Every fault that lives in a header window is still caught: readFrameHeader runs unchanged, so a bad prefix crc, a bad header crc, an out-of-range hash_len and an unknown flag kind all still void the open. Torn-tail detection survives intact -- a torn frame is a clean prefix, never damaged bytes, so offset + frame_len > total catches every truncation the payload and trailer short-read checks used to, using a length the header window already proved trustworthy. What is no longer caught at open is a bitflip in already-landed payload bytes, which becomes content()'s job; it crc-checks the frame it reads regardless.

openReadOnly -- the path every CLI verb reaches -- deliberately keeps .verify_payloads. No guarantee has moved on the production path yet, and none should until the explicit fsck verb exists to replace it. This half is what makes the cost measurable without taking on that decision.

The one test that changed is the one that had to: a corrupted payload on a complete frame no longer voids open. It now asserts the fault surfaces on the first content() read AND that openReadOnly still rejects the file eagerly. That second assertion is the tripwire for deleting verified_on_open without shipping the verb first. check-in: bae910f00e user: dmestas tags: logstore-open-cost

benchOpen timed LogStore.open -- the append path. Production never takes it: Repository.open falls back to openReadOnly, and Repository.logStore, the lazy content channel behind blob, cat, checkout and sync, calls openReadOnly with no cache in front. Every open number this effort has produced described a path no CLI verb enters.

That was survivable while the two paths ran the same scan. It stops being survivable the moment either one scans header-only, at which point the bench would report the optimized path while production still paid the full cost. --mode append|read-only defaults to append so existing invocations report unchanged numbers, and the mode is in the output line because the two are only comparable when a captured log can tell them apart.

Calibration on this tree, before any scan change: the modes agree within noise on all four shapes -- run B is 1% apart. That agreement is the check that the flag measures what it claims. check-in: e56a663b7a user: dmestas tags: logstore-open-cost

zig build test was 9m13s, and the tests it actually ran were a small fraction of that: it also compiled the e2e and smoke harnesses in the pinned ReleaseSafe and then never executed them. test is now unit tests only -- 2m54s cold, 1m42s warm -- and the new gate step carries those two compile proofs.

The dst, bench and smoke UNIT tests deliberately stay on test. They are unit tests despite their siblings' names, zig build dst/smoke run the simulation and the interop check rather than them, and CI invokes the lanes one by one and never invokes gate -- so moving them would have dropped them from CI while every step still looked green. ci_test's declared_steps records gate as intentionally not-in-CI, and names that failure mode for the next lane someone adds. check-in: 7363238ad5 user: dmestas tags: trunk

std.debug.assert is right for an internal invariant and wrong for a rule about data the function did not produce, and it is wrong in two different ways. Under the ReleaseSafe default it survives and PANICS: on the write path that is the trade we want, a crash over a durable bad frame, but on a peer-reachable path it is a remote denial of service. Under an explicitly chosen --release=fast it compiles to nothing and the input flows through unvalidated. One fix, a checked error, closes both.

Flags an assert only when all of: the fn is pub, returns an error union so a rejection was already available, the assert names one of its own parameters, and no earlier return in the body already rejects on that parameter. That last rule is what makes it signal -- delta.applyCopy returns MalformedDelta and then asserts to restate it, which is the CORRECT shape and must not be flagged.

The baseline is keyed on file, function and COUNT, never on line numbers: line numbers churn on every edit above them, while a count still catches a second bad assert added to a function that already has one. It fails in both directions, so a fixed finding must be removed from the list and the list can only shrink.

Existing findings are recorded, not fixed. Triage belongs in issue 19, where the ReleaseSafe default already reduces the write-path subset to crash-instead-of-corrupt and leaves the sync-side ones as the live exposure. check-in: 949e2b5542 user: dmestas tags: trunk

src/ carries 1086 std.debug.assert calls. They are how a detected inconsistency becomes a crash instead of a durable write -- TigerStyle's 'downgrade catastrophic correctness bugs into liveness bugs' -- and that trade exists only while the assertion does. ReleaseFast and ReleaseSmall compile every one to nothing, so a binary built that way converts each guarded corruption into an unguarded one, in a store where a bad frame is permanent and a crash is not.

Measured, not assumed. Read-side verbs are indistinguishable between the two modes: status, log and stat all sit inside noise on a 5 MB store. zit pack is not: 599ms fast against 15.2s safe, about 25x, because the delta encoder's best-of-N search is a tight per-byte loop where a bounds check is a large share of the work. That argues for setRuntimeSafety(false) on that measured loop -- local, reviewable, over code the fuzz and DST lanes already cover -- rather than stripping assertions from every other path in the binary. Not taken here.

Note the option surface changes: stating a preference replaces the -Doptimize enum with a -Drelease bool, so the deliberate escape hatch is the top-level --release=fast. -Doptimize=ReleaseFast no longer parses.

ci_test pins the preference, and says that if the project ever decides to ship unchecked, the pin should be deleted in that same commit so the decision is visible in history rather than arriving as drift. check-in: 4379187a10 user: dmestas tags: trunk

gatherCandidates capped the entries ADDED from the caller's priority slice, not the elements read: a duplicate hits ridListed and continues without advancing n, so the fill check never fires and the walk runs the caller's whole array with a linear dedup scan at each step. The only thing holding that back was an assert that the slice fits the window -- and std.debug.assert compiles to nothing in ReleaseFast and ReleaseSmall, so there was no guard in exactly the build where an unbounded scan would matter.

Slicing the prefix holds in every mode, and turns an over-length slice from a precondition violation into defined behavior: accepted, tail ignored. Both callers today are inside the window either way.

Two tests, both mutation-checked against the old shape. What is NOT pinned is the duplicate-heavy case: dedup leaves the surplus slots to the chronological walk, so the candidate set is identical either way and only the count of elements read differs. That is a cost property, not a behavioral one, and no test through this API can catch it -- the header comment says so rather than letting the pair read as covering it. check-in: bfec65b1a1 user: dmestas tags: trunk

A repository-local shun table (sync SS3.3), the zit shun verb over it, and the four suppression directions the drafts require: never requested, never announced or served, an arriving copy discarded, and the bytes reclaimed on demand through the existing atomic rewrite.

Built as eight slices, each through design review, TDD, spec audit, three review lenses, mutation proof and a full gate. Reading the eight as one change afterwards found what none of them could alone: a peer-reachable assert that crashed the client on a malformed DELTASRC, and a pre-rename guard that re-stat'ed a stale handle so two concurrent rewrites could un-shun each other.

Two defects predating the feature are fixed on the way. FileStorage.sizeImpl answers from a cached watermark, so pack's tail-revalidate compared a key against the number it was derived from and could never abort -- the guard had never worked. And a malformed file card could trap the client through PendingDeltaQueue.

Format version stays 1: an older binary meets an unknown frame kind and rejects cleanly, which is what it should do with a tombstone it cannot read. No interchange surface changed -- smoke covers ten fossil 2.28 scenarios including a fossil clone of a defer-shunned store and a real fossil push of a shunned artifact.

Known and recorded, not blocking: a running zit serve keeps serving the old inode until restarted after a --force reclaim swaps generations. check-in: e57bcceadf user: dmestas tags: trunk

Read as one change rather than eight slices, which surfaced defects no per-slice review could see.

A peer could crash us. ShunSet.contains asserted its key domain while its own vtable contract promised an out-of-domain name answers false and never traps -- and a 'file HASH zz SIZE' card reached it. It is a checked rejection now, and receiveDelta refuses a DELTASRC that is not a complete artifact name before one can mint a phantom, which also closes a pre-existing PendingDeltaQueue trap on the same card.

Two rewrites could un-shun each other. rewriteAndSwap's pre-rename guard re-stat'ed the caller's handle, which after a competing rename points at the old inode -- so the guard passed and the loser silently reverted the winner. It reopens .zit by name under the temp lock now, as shunAppendRows already did.

The client snapshot lacked the masked-basis guard the serve path has, so a kept delta rooted on a shunned base could be handed out in delta form.

shun_help denied two slices that landed after it was written: it still told operators a shunned artifact is only suppressed in one direction and can be re-announced back into the store.

Also: storestat's shun rows and bytes are wired into zit stat rather than left unused, so the printed breakdown still sums to the file size; the clone-send half is relabelled an erratum candidate against SS8.1/SS8.2 instead of claiming SS9.5, which is scoped to non-clone; and three stale doc bullets now describe the code that exists. closed check-in: d9578c6c6e user: dmestas tags: shun-implementation

A DST sweep that shuns artifacts (forcing a delta base, a manifest and a mid-chain artifact across seeds), reclaims through rewriteAndSwap, and cuts the run at sampled write/sync/rename boundaries. The oracle classifies the recovered store as the PRE image or the POST image and fails on anything else.

It earns its place: deleting packLog's pre-rename fsync is caught by this sweep and by nothing else in the suite. The concurrency-guard boundary is a second, DST-shaped pin on S3's fresh-stat fix rather than first coverage, and the module header says so.

Two oracle clauses are graded honestly as tripwires, not coverage. Surviving content is re-hashed by packLog before its own commit, so a defect that would corrupt it aborts the rewrite instead and the sweep just sees the PRE image; no mutant can make that clause fire. The PRE half of the third clause is subsumed by the byte-equality check. Both are kept cheap and labelled.

rewriteAndSwap and RewriteError are now pub: the property under test is a property of that sequence and reaching it needs the seam.

faulty_dir gains fileSetLength and fileClose wrappers, deliberately ungated so no sibling sweep's cut numbering shifts.

The default dst gate grows about 11s (~9%) for 12 sampled cuts per seed. check-in: cb22503707 user: dmestas tags: shun-implementation

.zit-dst-checkout-ws/ is the process directory the crash-recovery sweeps make their throwaway workspaces under. It is created at run start and removed by a defer at run end, so it is normally invisible -- but a killed or crashed sweep leaves the whole tree, .zit stores and all.

.zit-pack-tmp is the fixed-name temp pack and shun --force write the new generation into before the atomic rename; a run that dies between create and rename leaves one beside .zit.

Both are dotfiles, so a bare 'fossil extras' skips them and the tree reads clean while they sit there -- the same blind spot the .agent-config glob closed. Pinned by packaging_test. check-in: 9b95e3b9e5 user: dmestas tags: shun-implementation

SS7.7 L1434-1452, as one combined predicate: a target is shunned for admission when its complete name is in the shun table OR shun-sha1 applies to its SHA1 name. TargetAdmission.admits carries the whole rule, including the two card rules -- a clone file card BYPASSES the check and is stored; everything else consults it.

That asymmetry is deliberate and is the applied text. zit's own erratum argued the discard should apply in clone too and four reviews rejected it against fossil's xfer.c. A test pins the bypass so a later reader does not 'fix' it back into the shape that was refused.

For cfile the SS7.5 private discard runs first and the raw artifact-card count still increments -- both satisfied by placing the check below code that already existed, not by adding any.

Admission is target-only. A shunned DELTASRC is not independently rejected: a held basis still serves, and an unavailable one falls to ordinary dangling-delta handling whose phantom SS3.3 already excludes from gimme.

The planned e2e did not pin anything and was rewritten. A real zit push passes with the whole check removed, because transfer is receiver-driven and SS3.3 already stops this server begging for a shunned name. The shipped test POSTs an UNSOLICITED push+file container at a live serve -- the only shape that reaches this code -- and it goes red without it.

Smoke S10 is real interop coverage but cannot pin this slice for the same reason, and its header says so rather than letting a green be misread. check-in: 620ed6a9c3 user: dmestas tags: shun-implementation

LogIndexStore keeps a shunned holding in its enumeration and MASKS it rather than removing it, so a shun row appended between two rounds of one clone cannot renumber the survivors under the pager's cursor. held() answers false, the name leaves by_name, and SS9.5's 'igot for every eligible holding after the SS3.3 filters' is where that mask lives.

The plan overstated what was missing. S1's log-level mask already nulled content and wire, so a clone silently skipped a shunned artifact before this slice. What was actually broken is narrower and worse: because by_name still resolved the name, emitCloneReply's basis proof fired in a safety build and the SERVE PROCESS PANICKED -- a fossil clone got 'server did not reply', not an absent artifact. Reproduced against fossil 2.28 and pinned as smoke S9.

That also means the planned e2e ('a clone does not receive it') was never mutation-sensitive: it passes with this slice fully removed. The observable that pins the slice is the ABSENCE of an igot card naming a shunned artifact in a non-clone round's inventory walk, which now has its own test and a second round in the raw-HTTP e2e case.

Smoke uses a defer-shunned fixture, not the planned packed one: pack rebuilds the delta chains and so removes the exact hazard the check exists to catch.

logstore.zig carries a comment-only change: LogStore.shunned's doc claimed two questions were open that S4 and S5 have since closed. check-in: 230c2bbebe user: dmestas tags: shun-implementation

Refuses by default and names what would be destroyed; --force proceeds; never prompts. Reclaims by default -- one rewrite for N hashes, which is why the usage takes HASH... --defer records the tombstone without rewriting, and is also gated by --force: the row is irreversible and S1's mask hides the content immediately, so it is not the safe half of the verb.

The check-in this checkout is on cannot be shunned at all, with no override.

runShun does NOT reuse runPack's plumbing verbatim as planned -- it diverges on the filter (the source's deferred rows UNION the newly named hashes), on writing tombstones into the new generation, and on its renderer. The shared body is extracted as rewriteAndSwap instead.

Fixes a dead concurrency guard in that shared body, which predates this slice. FileStorage.sizeImpl returns a cached watermark rather than stat'ing, so validateTail was comparing a key against the number it was derived from and could never abort. pack carried the same dead guard. rewriteAndSwap now stats fresh at the check point; without that, this slice's claim that an aborted --force leaves no tombstone was unprovable.

Consequence lines carry no counts or filenames: 'N descendant check-ins' and a blob's path both need a reverse F-card walk over every manifest, which is too much for the cheap refusal path. Deferred, not faked.

No sub-manifest row: that type and its draft are not on this branch. shunConsequence is a total switch, so a ninth type will not compile without its diagnostic. check-in: be67814135 user: dmestas tags: shun-implementation

packLog takes a borrowed Filter; a dropped rid is not copied, PackStats.dropped counts it, and packVerifyShunned refuses to swap in a store still holding a digest the filter was told to remove. keep_all is the identity, so every other caller behaves as before.

The check runs BEFORE reconstruction and before the SS7.7 verify. A shunned artifact may be unlawful or simply broken, and verifying it would abort the very operation meant to remove it. Two tests pin the ordering: one frame whose content no longer hashes to its name, one that will not decode at all. The second closes a hole the first left in deriveCandidateHints.

putArtifactPacked's floor_span became optional. A delta whose base the filter dropped has no keepable frame, and 'I have no floor' is a different claim from 'my floor spans N bytes'; an in-band sentinel could not tell that from a miscomputed span.

pack_help no longer overclaims: damaged CONTENT does not block the reclaim, a damaged CHECKSUM still does, because the open-time pre-flight covers the whole store before any reclaim runs.

Merges S5's e2e block alongside this slice's. Fossil's suggested resolution would have dropped this one. check-in: 341009e699 user: dmestas tags: shun-implementation

sync SS3.3 L527-530 excludes shun-table names from the ordinary phantom scan. The predicate already existed and its call site hardcoded false; S5 supplies the durable state through a borrowed ShunSet whose .none identity leaves every existing construction site unchanged. The literal reading is kept: an igot for a shunned hash still CREATES the phantom (SS3.3 L511-514) -- it is simply never requested, so the phantom is inert and permanent.

Three consequences the reviews forced out:

A shunned phantom no longer counts as unresolved. It was told about and refused, not asked for and withheld; counting it made zit pull fail on every future pull, since the peer re-announces each round.

The server consults the set too. SS3.3 states the filter on the phantom scan without qualifying a side, and a server under push-receive is a receiver, so an inbound igot could otherwise walk a shunned artifact straight back in.

parseIgotCard now rejects a name that is not 40 or 64 hex, as parseGimmeCard already did (artifact-format SS3.2 L349-350). Widening the shun predicate to answer false for an out-of-domain name would have been the fail-open fix. check-in: 1c41f96e86 user: dmestas tags: shun-implementation

content, contentCached and wire return null for a name in the shun table even while its frame is on disk, so reclaiming the bytes later never changes what a verb prints. The guard sits in the one shared read body behind a ReadMask the call site names, so 'the masked set is exactly these three' is compiler-checked rather than asserted in prose.

It deliberately does NOT sit in contentAtOffset. sync SS7.7 L1445-1450 lets a held-but-shunned DELTASRC serve as a delta basis, and zit satisfies that for free because delta_of is a byte offset -- base resolution never re-enters the rid-keyed doors. A test pins it so a future tidy-up downward fails loudly.

pack reads through an unmasked maintenance door: in this slice a shunned frame still copies forward, since dropping it is the S2 filter. pickBase probes through the masked door, so a shunned frame never roots a new delta. check-in: 3e5895dfb7 user: dmestas tags: shun-implementation

FrameKind.shun carries a tombstone keyed by the shunned artifact's whole digest (sync SS3.3 L518-525: persistent, repository-local, keyed by complete artifact name). Never interned, so lookup() still means 'the content frame for this hash'; scan rebuilds the set. putShun is idempotent and returns BadKey for any width but 20 or 32 -- checked, not asserted, so a ReleaseFast build cannot append a durable un-openable frame.

pack carries the table forward. Without that the repo's own compaction verb silently drops every tombstone, which would make this slice's durability claim false the first time anyone packs. The pack FILTER is still S2.

storestat and the benchmark restate FrameKind's universe; both now fold from the enum so a future kind fails to compile rather than vanish from the byte accounting. check-in: ce40b2526e user: dmestas tags: shun-implementation

Trunk carried the pre-errata draft. The corrected text -- the SS3.3 shun-table definition and the SS7.7 receive-side shun admission rule -- was only ever committed on the sub-manifest branch, so any branch cut from trunk was building against superseded normative text. This is the file the errata records name as applied (sha256 068d7672...). check-in: d186a44ea3 user: dmestas tags: trunk

.agent-config/ collects per-session tooling traces at the repo root. Being dotfiles they never appeared in a bare 'fossil extras', so they were held out of commits by fossil's default alone rather than by any rule -- one 'addremove --dotfiles' would land them permanently. CLAUDE.local.md is checkout-local operator config. Both are globbed now and pinned by packaging_test. check-in: a27cf4b4a9 user: dmestas tags: trunk

std.hash.Crc32 is byte-at-a-time over one table (~350 MB/s here) and the open-time scan CRCs every payload byte, so it was ~93% of a cold open. Slicing-by-8 in its place: a 416 MB store opens in 292ms against 1.22s, warm reads run 3.3x, the write path 1.5x. Bit-identical by construction, so no format changes and no repository is affected. check-in: 602c2459f2 user: dmestas tags: trunk

Both consumers live in their own build roots and cannot reach crc32.zig by path, so the export is forced. Adds the differential fuzz lane against std — the only coverage that can see a self-consistent but numerically shifted implementation, which would orphan every existing .zit file.

The implementation brief prohibited this export, citing repocache as the precedent for unexported leaf modules. That precedent does not apply: repocache has no out-of-module consumer. storestat and repoimage, which do, are both exported. Overridden deliberately. closed check-in: c57c16c5cd user: dmestas tags: logstore-index

The other three on-disk formats defined in terms of this checksum: the cache image self-checksum, the bisect session trailer, the undo journal trailer. Same swap, same values. check-in: 55d6e7c7bb user: dmestas tags: logstore-index

The four frame checksums plus the file header. The open-time scan CRCs every payload byte, so this is ~93% of a cold open: 416 MB goes 1.22s to 292ms. No format change — the values are identical by construction. check-in: dd71dd3006 user: dmestas tags: logstore-index

Bit-identical to std.hash.Crc32, which every on-disk checksum in the store is already defined in terms of. std's is byte-at-a-time over one table (~350 MB/s here); this folds eight bytes per step. Nothing uses it yet. check-in: aef8f539b5 user: dmestas tags: logstore-index

A serving release over 0.1.2. Where 0.1.2 made a large repository cloneable, 0.1.3 makes serving one fast and lean.

A server against a large repository comes up immediately: the listener opens before the repository is read, so a client queues in the accept backlog while the server primes rather than being refused during a startup a large store made long. A push no longer stalls inside its own reply deadline; the snapshot rebuild it triggers now runs between connections. Serving reads the repository through an on-demand log-offset index rather than a resident copy, so peak memory to serve a 67,499-artifact repository fell from about 157 MB to about 32 MB, with clones byte-identical to before.

Also: the end-to-end suite is safe to run concurrently with itself (fresh loopback ports), and the unclassifiable-artifact tolerance rule is now covered by deterministic simulation and an end-to-end scenario. check-in: a469147254 user: dmestas tags: trunk, version-0.1.3

The rule that lets state assembly skip a stored artifact it cannot classify — when the artifact's bytes verify against the name it is stored under — shipped with the fix but was pinned only by two unit tests over hand-built logs, and by a manual clone of the real fossil repository. Neither the simulator nor the end-to-end suite exercised it, so a regression could pass every automated gate.

Deterministic simulation grows an artifact-level generator (src/sim/ assembly_sweep.zig): it emits artifacts whose card sets sit in and around the required-card table — some verifying against their stored name, some stored under a wrong one — and asserts the split directly, that assembly completes iff every unclassifiable artifact verifies, derives no state from one, and stays fatal on a single artifact whose bytes do not match its name. It runs every seed. assembleState is made public for it to drive; its body is unchanged.

The e2e suite gains a scenario that clones a served store carrying an anonymous wiki artifact (cards D L W Z, no U — the shape that first surfaced this) and asserts update tolerates it, with the read verbs and assembly agreeing on the same store. Reverting the verify-then-skip arm to an unconditional failure reddens the sweep, the e2e scenario, and both unit tests. check-in: c2851f933e user: dmestas tags: trunk

Priming a serve snapshot copied every artifact's container out of the log into an in-memory store and held them there for the server's lifetime. The log had already been scanned and every frame CRC-checked to build the rid table, so the copy was a second full read whose only product was resident bytes: on a 254k-artifact repository it was two thirds of the prime time and a few hundred megabytes of RSS, all to re-hold what the log already holds.

The snapshot is now a log-backed store: an index of hash -> log location, and container bytes read from the log on demand as a round emits them, memoised only for the round and released after it. Persisted order is preserved so the clone pager still cursors correctly, the binary-digest to hex-wire-name conversion stays at its one site, and a read fault after the scan surfaces as a server storage error so a clone is never silently truncated. The old whole-store copy is retained only where it is actually needed: the pull client's holdings and the equivalence test's oracle.

Measured, 67499-artifact repository, ReleaseSafe: a clone off the indexed serve is byte-identical to a clone off the copy-based serve (both materialise all 983 files of trunk identically), while serve peak RSS falls from 157 MB to 32 MB. The one remaining O(store) cost, the startup scan itself, is a separate change. check-in: e17845e40d user: dmestas tags: trunk

zit serve built its whole in-memory snapshot of the repository before it opened the listener socket. The build is O(store), so past a few thousand artifacts it outlasted the client's readiness probe and the client reported the server never came up — the store could be served, just never reached.

Bring-up now binds first: listen, announce, then prime, then accept. A client that connects during the prime queues in the kernel accept backlog instead of being refused, so the readiness window starts at the bind rather than after the scans. The listener open moves ahead of BOTH O(store) reads on the path — the config read and the prime. The connection deadline is armed only on accept, so priming never falls inside it. On a 254k-artifact store a connect now completes in ~2 ms against a ~5 s prime.

A push made the snapshot stale, and the old code rebuilt it inside the pushing round's own reply deadline — an O(store) rebuild charged to the request that caused it. The rebuild moves to a maintain hook the accept loop runs between connections, with no connection open to charge it to; the append-only stat gate makes the stale snapshot self-correcting until then.

The reorder is inherently a large-corpus latency property: reverting it leaves every unit test green, because the failure only shows when prime latency exceeds the readiness window. It is pinned in the milestone tier instead, where smoke already lives: zig build bench -- serve synthesizes a store, spawns the real server, and asserts time-to-first-connect is a small fraction of the prime (fixed order ~0.1; reverted order jumps past 1.0). The de-race and the accept-loop hook are pinned by ordinary unit tests. check-in: de218fa6f6 user: dmestas tags: trunk

The 'cli push' and disconnect-robustness scenarios bound a fixed loopback port, so two e2e binaries running at once — the ordinary case under a parallel build or a busy CI runner — collided on it and one push exited non-zero, failing an unrelated-looking assertion. The failure was invisible because the assertion reported only the child's exit code and discarded its stderr.

Each scenario now draws a fresh port from the kernel (bind :0, read back the assigned port, release) and waits for the child to accept before driving it, the same approach freshLoopbackPort takes for the Fossil smoke tests. The push assertion now surfaces the child's stderr on failure, so the next divergence is diagnosable rather than merely counted.

Verified on Linux under the exact condition that reproduced the flake: two 'zig build e2e' runs in parallel, ten rounds, x86_64 and aarch64 — 20 of 20 green on each, where the fixed-port version failed intermittently. check-in: b4a5886252 user: dmestas tags: trunk

Regenerated from the RFC suite after the erratum repo-state-00-s4-unclassifiable-artifact was applied. New §4.4 states what assembly does with a stored artifact classification rejects: an artifact whose octets hash to the name it is stored under is opaque to repository state and MUST NOT fail the assembly, while one whose octets do not hash to its name is a distinct integrity failure and MUST be rejected. The remaining §4 subsections renumber behind it.

The draft was previously silent on this, and that silence was the trap: an implementation conforming to both drafts could reject an artifact per the required-card rule and then treat that rejection as fatal, which made the canonical fossil repository unassemblable. src/update.zig implements the rule this text now carries. check-in: 42e1faa9fc user: dmestas tags: trunk, version-0.1.2

The 0.1.2 release check-in carried the version bump and the README pointer but not the notes themselves: the add that would have tracked them was refused by a locked checkout database, and the refusal was hidden behind a pipe that kept only the last line of output. The file existed on disk and in no commit. check-in: aabc10268a user: dmestas tags: trunk

Two defects made zit unusable against a repository of real size and real history. Serving a sync round cost work proportional to the served store rather than the request, so past a few thousand artifacts a round outran the connection deadline and the server answered nothing at all. Repository-state assembly rejected an entire repository over one stored artifact it declined to classify, while every read verb skipped the same artifact.

Together they meant a clone of the canonical fossil repository could be fetched but not used. It now clones in 67 rounds and materialises all 983 files of the check-in, each byte-identical to fossil's own checkout of it. check-in: 800caecbf2 user: dmestas tags: trunk

A conforming repository contains artifacts a conforming parser declines, and assembly rejected the whole repository over one of them while every read verb skipped it. Bytes that hash to the name they are stored under are stored correctly and merely unclassifiable, so they are opaque to state and skipped; bytes that do not are an integrity failure and stay fatal.

This closes the tolerance half of the update-diagnostics work. On a clone of the canonical fossil repository, update now materialises all 983 files byte-identical to fossil's own checkout. check-in: 6a0f1f3f66 user: dmestas tags: trunk

A conforming repository contains artifacts a conforming parser declines. The artifact grammar requires the U card on a wiki artifact; fossil elides it when the submitter is "nobody", stores the result and syncs it. A clone of the canonical fossil repository therefore carries such artifacts, and assembly rejected the whole repository over one of them while timeline, ls, info and branch read the same store correctly.

The two conditions are separated by the name. Bytes that hash to the name they are stored under arrived and were stored correctly, so the only event is that this reader's grammar does not admit them; the artifact is opaque to state and is skipped, exactly as a non-participating type already is. Bytes that do not hash to their name are an integrity failure and stay fatal. Verification runs before the skip, which is what keeps the second from being swallowed by the first.

On a clone of the fossil repository, 67499 artifacts: update went from exit 1 having materialised nothing, to materialising all 983 files byte-identical to fossil's own checkout of the same check-in. closed check-in: f4736670a8 user: dmestas tags: fix10-tolerance

Two costs made a clone of a large repository O(rounds x store) of work; past a few thousand artifacts a round outran the connection deadline and the server answered nothing at all.

The reply's advertise walk classified every rid by calling content(), which INFLATES a wire-backed entry, so announcing the store cost its uncompressed size every round. It now reads held(), answering from the retained form alone. The send gate still reads content(): uploading composes from plaintext.

ServeCtx caches the snapshot and rebuilds only when a stat shows the log grew, so a commit made alongside a live server is still advertised next request. That rebuild is O(store) and stays out of every client's budget: serve primes before it listens, and a round that appends rebuilds before returning. Reuse needs isolation, since the responder interns phantoms as it answers, so a new OverlayStore gives each round a write overlay. Inflating memoises, so a snapshot outliving its round accumulates the whole repository's plaintext; it is released per round.

The one conflict, at the tail of src/cmd/sync.zig, was two independent test appends; resolved as the union. check-in: 6763d35da7 user: dmestas tags: trunk

update, open, merge and bisect collapsed ApplyError.StateRecordInvalid and ApplyError.Corrupt onto one line, "cannot read repository: repository is corrupt". Corrupt means stored bytes do not match the name they are stored under. StateRecordInvalid means only that a stored artifact was not usable as a check-in state record, which says nothing about integrity. An intact fossil clone reaches the second routinely, so update accused an undamaged clone of corruption while timeline read the same store fine.

StateRecordInvalid now carries its own reason text, shared across the four verbs so they cannot drift or re-merge. The bisect mapper had a blanket else arm swallowing the whole set into the corruption line; it is now exhaustive.

Diagnostics only: assembly still rejects these artifacts, so update still exits 1 on such a clone. The tolerance fix is separate; the shape that triggers it, an anonymous wiki page with no U card, is pinned. check-in: e3bbb4ff22 user: dmestas tags: trunk

Two costs made a clone of a large repository O(rounds x store) of work; past a few thousand artifacts a round outran the connection deadline and the server answered nothing at all.

The reply's advertise walk classified every rid by calling content(), which INFLATES a wire-backed entry, so announcing the store cost its uncompressed size every round. It now reads held(), answering from the retained form alone. The send gate still reads content(): uploading composes from plaintext.

ServeCtx caches the snapshot and rebuilds only when a stat shows the log grew, so a commit made alongside a live server is still advertised next request. That rebuild is O(store) and stays out of every client's budget: serve primes before it listens, and a round that appends rebuilds before returning. Reuse needs isolation, since the responder interns phantoms as it answers, so a new OverlayStore gives each round a write overlay.

Inflating memoises, so a snapshot outliving its round accumulates the whole repository's plaintext. It is released per round. Four clones of 3001 artifacts, ReleaseFast: 12 MB and 0.09 s, against 26 MB and 0.15 s. closed check-in: e2e9233582 user: dmestas tags: fix9-serve-stall

zit update, open, merge and bisect collapsed ApplyError.StateRecordInvalid and ApplyError.Corrupt onto one line, "cannot read repository: repository is corrupt". They are different claims. Corrupt means stored bytes do not match the name they are stored under. StateRecordInvalid means only that a stored artifact was not usable as a check-in state record, which says nothing about integrity.

An intact fossil clone reaches the second routinely: fossil stores and syncs artifacts it declines to crosslink itself. Assembly treats those as fatal while the read verbs skip them, so update accused an undamaged clone of corruption while timeline read the same store fine.

StateRecordInvalid now carries its own reason text, shared across the four verbs so they cannot drift or re-merge. The bisect mapper had a blanket else arm swallowing the whole set into that corruption line; it is now exhaustive over ApplyError.

Diagnostics only: assembly still rejects these artifacts, so update still exits 1 on such a clone. The tolerance fix is separate; the shape that triggers it, an anonymous wiki page with no U card, is pinned. closed check-in: f75dfdfbe7 user: dmestas tags: fix10-update-diag

zit serve rebuilt its whole in-memory snapshot from the log on every request, and classified held-vs-phantom by calling content on every rid, forcing a full inflate of the entire repository per request. A clone cost O(rounds x store) of server work; past a few thousand artifacts one round outran the connection deadline and the server answered nothing at all.

ArtifactStore grows heldCount, the aggregate of the existing non-inflating held predicate, and serve counts with it rather than re-deriving a worse predicate caller-side. ServeCtx caches the snapshot and rebuilds only when a stat shows the append-only log grew, so a commit made alongside a live server is still advertised on the next request.

Reuse needs isolation: the responder interns phantoms and upgrades entries as it answers, and that would change what later rounds serve. A new OverlayStore answers each request over a per-request write overlay, reads falling through, writes confined to the overlay. The persist pass now takes an ArtifactStore.

On 67,499 artifacts: unfixed, no round completes; fixed, one snapshot build then sub-millisecond rounds. check-in: 2874978461 user: dmestas tags: fix9-serve-stall

Harden native Linux filesystem behavior, deterministic repository iteration, update and undo transitions, Fossil smoke-process containment, and the three-platform CI matrix. check-in: be97df0398 user: dmestas tags: trunk, version-0.1.1

The limitations table stopped short of where a reader would look for more detail; it now points at the release notes rather than growing into a second copy of them. The named build steps were missing zig build cov, the kcov coverage report over the test suite. check-in: 7f7828b7d6 user: dmestas tags: trunk, version-0.1.0

The notes named the fossil version smoke passed against, which is a legitimate conformance fact, but then explained CI runner provisioning and told a future editor to reconcile it against a workflow log. That is tooling detail with no place in a document meant to outlive the tooling. The fossil version stands alone as the fact that matters. check-in: 07c840ae4b user: dmestas tags: trunk

The package version and the version the binary prints move from the 0.0.0 development placeholder to 0.1.0, the first tagged release. Both sites carry the same string so the manifest and the running binary never disagree about what this is. check-in: 9e0be2fc75 user: dmestas tags: trunk

The generator only ever emitted strictly increasing, unique modification times, so the equal-second corner -- where two check-ins tie and the name must resolve by DAG ancestry, not hash order -- was covered only by hand-written regression tests. That corner was a real bug once. A new tie mode reuses the previous mtime on a seeded, bounded roll and shapes a reused-mtime check-in as a descendant chain, the exact shape the bug took; the oracle resolves a tie the way production does, by ancestry and then least hash, so a resolver that fell back to hash order is caught. Order comparison is done up to that tie-break, with the two genuinely implementation-defined dimensions skipped under ties and documented. Reverting the real tie-break in a scratch build makes the tie sweep fail where the unique sweep stays green -- proof the simulator now sees the class it was blind to. Default seeds draw no extra entropy, so the existing universe is unchanged. check-in: b354fb5947 user: dmestas tags: trunk

sim/gen.zig emitted strictly-increasing unique mtimes, so the same-second selection tie -- once a real defect, the B1 regression cluster -- was only example-tested, blind to a class that already bit once.

gen.zig gains a tie-permitting mode: a seeded roll reuses the previous mtime, bounded to short runs, and each reused-mtime check-in is shaped as a linear child of the previous one, so an equal-mtime run is a real descendant chain. oracle.zig now computes the documented tie-break in resolveName, tipSlot and openLeafOf -- descendant-before-ancestor over full ancestry, then least artifact hash (repostate.zig ancestryMaximal) -- so a resolution reverting to hash order is caught, not mirrored.

dst.zig runs the tie sweep as additional coverage; the unique sweep is unchanged. Order-independence keeps its total-rule dimensions (leaf set, name resolution) under ties and skips the open-leaf set, whose equal-mtime supersede is implementation-defined (Section 5.3); the Section 6.3 branch representative is first-seen under a tie, checked in the unique sweep only.

Reverting the tie-break to hash order trips the tie sweep at seed 1. closed check-in: c52cc8cc76 user: dmestas tags: h10-ties

Phase 2 only ever proved convergence after the network was fully healed, so a protocol that converges on a perfect wire but livelocks under sustained loss would have passed. A new degraded phase holds a fixed, never-healed drop and duplication rate on both links and drives a fresh session through the same safety, convergence and termination oracles, within a window scaled to the loss rate: a round makes progress only if both its request and reply arrive, so the expected round count grows as one over the delivered-fraction squared, times a safety factor, with the derivation in a recomputable comment. It converges with wide margin -- across many seeds the worst run finished in a small fraction of the window at every rate tested -- so there is no livelock. Pinning the window below the real need trips the oracle, so the check is not vacuous. The gate runs one modest rate; the soak sweeps a range. check-in: b8ef4a68e5 user: dmestas tags: trunk

The loss-scaled convergence window used divCeil with a catch unreachable for the divide-by-zero arm, which the precommit guard flags outside tests. The delivered-percent denominator is provably at least one, so plain integer ceil division gives the same bound with no error union. closed check-in: 0fbc9225bf user: dmestas tags: h7-degraded