Timeline

undo's per-file cap read managed.max_file_bytes -- the serialized PATH-TABLE bound (64 MiB) -- as if it were a content ceiling. Any tracked file over 64 MiB (commit allows 1 GiB) failed captureFile, and since a durable pre-image gates any mutation, that aborted checkout/update/merge outright.

Per-file cap is now max_tracked_file_bytes, homed in managed so every layer shares one number. Journal read sizes from stat instead of a derived cap that would have refused journals record can write. (#34) check-in: 1cc2d055b9 user: dmestas tags: sub-manifest

A total-round cap multiplied against the per-round batch size into an accidental ceiling on repository size (~1M artifacts), which a healthy pkgsrc-scale store tripped while converging every round. Guard now counts consecutive rounds that moved no bytes; announcement is not delivery, so a peer announcing phantoms forever still trips it.

Also: import-git reports sub-manifest memo hit rate and whether the retention budget latched (#35). check-in: 6682409ff0 user: dmestas tags: sub-manifest

A citation may carry several ranges: `§9.3 L3278/L3320-3322`, `§7.7 L1465/L1469-1471`, `§4.6.4-m6 (L1014)`. Matching only `§SEC Lnnn` saw the first and was blind to the rest, so half a compound citation could be re-keyed while the other half stayed rotted -- on the same line, looking equally verified. An adversarial reviewer caught that during a re-key pass; the tool could not have.

342 line references sat outside the checker's view. Each now binds to the NEAREST PRECEDING section on its own line, which is the rule that makes this safe: a range written BEFORE any section on the line -- `L545-547 hands clone to §8.1-§8.4, and §8.2 L1504-1506` -- belongs to earlier prose, not to a section further right, and is left alone rather than misattributed. 122 became checkable; 12 leading ranges were correctly skipped. Ranges are never carried across a line break for the same reason.

This also surfaced §10.5, which the old pattern could not see at all because a requirement id sits between section and range (`§10.5 r43 L2477-2479`). §10 is Error Handling with only §10.1; the send-private and client-url claims are §9.5 Private-Content and Discovery Pragmas (L1983-1985: "send-private enables private sync only with x ... Clients repeat it every private round"). Same -1 downshift already verified for §10.3 and §10.4. 21 sites renumbered, and not-a-section is back to 0.

Baseline 80 to 173: 93 defects that were always there and are now visible. The population it measures grew 821 to 943.

test 3495/3495, fmt. Issue #18. check-in: 43c89e841f user: dmestas tags: sub-manifest

d4e0e0d6da fixed six renumbered sections and deliberately left their line ranges stale, on the reasoning that a correct section turns a stale range into an ordinary lookup. This is that lookup: 20 of 100 survived review.

The 20 percent hit rate is lower than the pass before it, and the reason is worth recording. It is NOT that the renumbering targets were wrong -- §11.1 L2604-2607 reads identically before and after that commit, so those citations were never renumbered by it. What the pass found instead is a second, older layer of mis-sectioning underneath the first: claims about server-emitted error cards citing §11.1 Credential Replay when they belong in §10.1 Error Card. Fixing one generation of drift exposed the next.

Two refusals name failure modes the applier could not have caught. One proposal re-keyed half of a COMPOUND citation -- sync.zig:961 carries `§9.3 L3278/L3320-3322` -- which would have welded a freshly verified range to a still-rotted one on the same line. Another submitted a quote that stopped mid-word, with the half of the claim it needed sitting in the sentence it cut off.

The 66 unresolved are now the honest core: three independent adversarial passes have each declined to invent an answer for them.

Baseline 100 to 80; 90 percent of citations now resolve. test 3495/3495, fmt. Issue #18. check-in: dcd6c48df9 user: dmestas tags: sub-manifest

Issue #18: "a reference to a plan must not wear § notation." 80 sites across eight files cited `§E`, meaning item E of storage-track-plan.md -- a document that is not a draft and is not even in this repository. They read exactly like spec citations, including in test names.

`§E` is unambiguous: no draft defines an Appendix E, and every site sits in the storage family (checkout, logstore, managed, pack) whose headers already name storage-track-plan.md. Bare references become `plan E`; where the filename is already present it reads `storage-track-plan.md item E`, which says the same thing without the false notation.

One carried a line number, `§E L119`. A line cite into a file outside the repository cannot be verified by anything, so the range went with it.

That was the last `not a section of any draft` in the tree: 1 to 0. Baseline 101 to 100.

Deliberately NOT swept: §A and §B, which are real draft appendices -- §A alone has 162 subsection citations, and a blanket letter sweep would have destroyed every one. §C (78 sites) is left for a separate ruling: it looks like the same plan, but Appendix C exists in three drafts, so it needs the discriminator applied per site rather than in bulk.

test 3495/3495, fmt. Issue #18. check-in: ee4e0e623a user: dmestas tags: sub-manifest

Two comment families asserted RFC requirements the drafts do not make.

The 65535-byte line bound is ours, not the protocol's. §5.2 L962 says "This protocol imposes no separate maximum line length" and §11.3 L2199 repeats it, yet four tests named it an RFC bound and max_line_len's doc cited §5.2 as its source. The constant stays -- a finite receive window is a resource limit, and §11.3 L2200-2201 leaves payload limits to "declared counts and storage policy" -- but it is now documented as local policy, including that a conforming peer may legally send a longer line and we refuse it. Issue #32 justified this constant as a principled tripwire on the strength of that citation; the doc now records that the justification was void.

The §3.10 comments reasoned about a section that no longer exists -- "§3.10 restates §3.2 and defers to it, so introduces no new order rule". The argument had no subject. Each test asserts real behaviour and is kept; the citations now name the section that actually governs it, and r1 points at §3.2 L376-377, the rule the deleted section deferred TO.

not-a-section 6 to 1. Baseline 106 to 101. test 3495 (1 environmental e2e skip, no failures), fmt. Issue #18. check-in: 8aa2cf2c35 user: dmestas tags: sub-manifest

Second pass over the leftovers after the bulk re-key. 29 of 107 survived review; 10 were refuted and 68 returned unresolved, which is the right shape for leftovers -- the easy ones were already gone.

Most are Appendix A worked-example citations, now visible to the checker for the first time: the round-1/round-2 request and reply bodies, their igot and gimme runs, and the SHA3 gate response. An example transcript is often the better citation for a claim about concrete wire bytes than the prose that governs them.

The refusals stay instructive. One proposal was rejected because the directional half of its claim lives in §3.2.1 Round Message Layout rather than §6.1, so it was mis-sectioned rather than mis-lined; another because its range stopped mid-sentence, dropping the words the claim rested on.

Baseline 135 to 106. test 3495/3495, fmt. Issue #18. check-in: 22d23eae1c user: dmestas tags: sub-manifest

Six cited sections do not exist in the drafts at all, so no line range could ever have been right. Each was renumbered, and each successor is confirmed by subject match against the section's own text:

§B.4 -> §B Appendix B flattened; cookie-card, url-login-value, pct-user, unreserved and pct-encoded all still exist, every citation exactly 1787 lines above its production §12.1 -> §11.1 Credential Replay §12.2 -> §11.2 Private-Content Exposure and Protected Transport §10.3 -> §9.3 Configuration Exchange -- "the server MUST require Admin; otherwise emit error not authorized to push" §10.4 -> §9.4 Commit-Lock Pragmas -- every site is ci-lock/ci-unlock §9.1.1 -> §8.5.1 Unversioned-File Synchronisation -- "status 3+MTIME-CMP (2 older, 3 identical, 4 newer)"

The §B citations carry corrected line numbers, each asserted to land on the production its comment names. The rest carry the section only: once the section is right the stale range is an ordinary "outside" case, which the re-key pass already handles, and inventing a range here would be the clamping this work exists to prevent.

Deliberately untouched, all needing a decision rather than a lookup: §12.3 (successor §11.3 states the OPPOSITE -- "This protocol defines no separate per-line byte limit before LF" -- while four tests assert a 65535-byte bound as an RFC requirement), §3.10 (removed; the comment reasons about the section's own existence), and §E (a reference to storage-track-plan.md wearing §-notation, which the issue forbids).

not-a-section 36 to 6. Baseline 144 to 135. test 3495/3495, fmt. Issue #18. check-in: d4e0e0d6da user: dmestas tags: sub-manifest

Of 475 candidates, 368 were located by reading the draft text that supports what each comment claims and survived adversarial review; 23 were refuted and 84 returned unresolved. sync.zig and sync_test.zig now resolve at 86 percent, up from 39.

The refusals are the useful part. Several proposals verified byte-for-byte and still pointed at the wrong sentence; one range truncated away the half that carried the claim; one cited text that REFUTES its comment. The unresolved are mostly mis-sectioned rather than mis-lined -- §9.1 cited for push/pull card grammar, §10.1 for the client-version pragma -- and one where an erratum inverted the claim, so the COMMENT is now wrong and no line range would fix it. Those are left for a section decision.

The checker also could not see `§A.1`-style citations at all: 36 of them, unverified while looking checked, 25 of them broken. Appendix headers use `Appendix A. Title` and `A.1. Title`, so both forms are recognised now.

Baseline 477 to 144, over a population that grew 781 to 817.

test 3495/3495, fmt. Issue #18. check-in: 18e5fa7bbb user: dmestas tags: sub-manifest

All four assert that the artifact hash names the uncompressed, fully reconstructed bytes, and cited §4.1 for it. No §4.1 in any of the five drafts says that -- §4.1 defines the container. The statement is sync §7.7 L1465, "Integrity hashes reconstructed artifact bytes, not wire payload."

These were the ones the re-key pass returned unresolved rather than guessing at: wrong section, not drifted lines, so no line range would have been right. Naming the draft because §7.7 also exists in artifact-format.

Baseline 481 to 477. test 3495/3495, fmt. Issue #18. check-in: f1e10128f8 user: dmestas tags: sub-manifest

Each was located by reading the draft text that supports what the code comment actually claims, then adversarially re-checked against a verbatim quote of those lines. Clamping a range into its section would satisfy the checker while pointing at the wrong paragraph, so the checker alone was never the gate.

Of 33 proposals, 2 were refuted on exactly that ground -- the quote verified byte-for-byte and still did not state the claim -- and 4 came back unresolved because no draft's §4.1 states what those comments assert at all. Those are mis-sectioned, not line-rotted, and need a section decision rather than a re-key.

Also restores container.zig's `decode` doc block, which 7d26a31305 left stranded above `decodePrefix` when that function was inserted between the comment and the function it describes.

test 3495/3495, fmt. Issue #18. check-in: e5db38a6df user: dmestas tags: sub-manifest

A bare `§4.1` exists in five drafts, and the report named whichever was witnessed first -- so container.zig's container-format citations were reported against submanifests when they mean sync-protocol. The verdict was right and the message sent a re-keyer to the wrong document.

The report IS the worklist, so that is a defect. Baseline unchanged. check-in: 2fa7460b8f user: dmestas tags: sub-manifest

Most `§X.Y Lnnn` citations no longer point at the text they name. Line numbers rot each time a draft is re-rendered for an erratum, and a citation resolving to the wrong paragraph is worse than none: it looks authoritative.

scripts/check-citations.py parses section spans out of each draft and asserts every cited range falls inside the section it names. It reproduces the survey in #18 -- 65 sections-not-found exactly, 674 vs 667 checked, the delta being citations a wider regex catches.

It splits on "\n" rather than using splitlines(), because drafts carry form-feed page separators that splitlines() also breaks on, drifting every line number further the deeper into the draft it sits. That bug makes correct citations look broken, and it is why this is a script with a reproduced baseline rather than a one-off measurement.

CI pins the count at 512 and fails in BOTH directions, so a new broken citation and a fixed-but-unlowered baseline are each caught. Re-keying the backlog stays separate work; this stops it rotting further.

Also drops parseGimmeCard's assert restating the check above it. parseIgotCard's remaining assert tests a different property and stays.

test 3495/3495, fmt. Issue #18. check-in: 0f80982412 user: dmestas tags: sub-manifest

verifyCommits flattened each check-in's whole S-tree against the published store through a resolver that died with the call, so a node near the root was re-fetched, re-inflated and re-parsed once per descendant check-in -- the cost a89b63ad05 removed from deriveCandidateHints, still present here.

repostate.SubManifestMemo already does this for Repository.flattenIfTree, and documents rebinding `inner` for reuse across walks; the log-only path had no way to hold one. flattenTreeFromLogMemo takes a caller-owned slot and the Verifier keeps one for the whole pass.

Byte-identical output. The win is commit count times inter-commit node sharing: 121 commits over 12,000 files touching 6 per commit, 9.88s to 8.19s. Wide per-commit churn destroys the sharing and is unchanged, never slower.

test 3495/3495, fmt. Issue #33. check-in: c120b93af5 user: dmestas tags: sub-manifest

deriveCandidateHints visited every artifact in the store and reconstructed each one in full, only for parseForImport to discard the blobs. Two bytes decide it instead: §4.4 makes byte 0 of a structural artifact a card letter and byte 1 its delimiter, so a prefix inflate classifies without paying for the whole artifact.

Conservative both ways -- a delta frame or a prefix error falls through to the full read, and only a definite mismatch rejects -- so output is byte-identical, verified on every corpus tried.

Pack of a full-form-dominated store (the shape import-git's automatic pack reads) 10.84s to 8.49s. Neutral on an already-packed store, where deltas dominate and cannot be classified this way.

test 3495/3495, fmt. Issue #32. check-in: 7d26a31305 user: dmestas tags: sub-manifest

Issue #32. `zit stat` gains full-form-bytes and delta-bytes, a partition of content-bytes. The counts alone mislead: a real history is 23% full-form by artifact and 84% by byte, because an artifact is stored whole exactly when delta did not pay.

tree_cache_bytes was 1 MiB, smaller than a wide tree's root fan-out, so nodes were evicted and re-decoded on the next commit that touched them. 64 MiB: byte-identical output, 67.4s to 56.5s importing 45,000 files. Unmeasurable below the old budget, which is why it went unnoticed.

fast_export.Limits.max_events loses its 1,000,000 default rather than gaining a new one. #29 removed that cap at the CLI and left it armed here; six callers silently re-inherited it, the benchmark among them.

test 3495/3495, fmt. check-in: 367539bec4 user: dmestas tags: sub-manifest

The 3,000,000 retained-entry cap and the 1,000,000 event cap both bound by proxy with a round number attached, and each stopped legitimate imports before the real bound was reached. Bytes already bound what costs: max_retained_bytes for retained state, max_total_data for payload. Deleting max_export_events alone was a no-op because the reader carries the same default, so it is now passed maxInt explicitly.

test 3495/3495, gate, e2e, git-import-cli, git-import-e2e. Closes #29. check-in: 255b8f2489 user: dmestas tags: sub-manifest

Issues #27 and #28.

deriveCandidateHints walked each check-in's whole S-card tree and its parent's, for every check-in. Shared subtrees can never pair -- an unchanged node has the same rid on both sides, so pairAgainstParent skips it -- and their blobs were already recorded at introduction, which Pass 2 relies on only as a distinct set (it compacts with dedupSortedRids). So the walk now descends lockstep and prunes any subtree whose S node is the same artifact in both. O(check-ins x tree) becomes O(check-ins x changed paths): sub-manifest loads drop 56 percent on libfossil, to near the distinct-node floor, and the ratio grows with tree size.

The visits that remain were re-inflating. contentOfEntry memoized only .delta frames, so a .zlib sub-manifest re-inflated on every visit -- which is why a pack profile is dominated by flate.Decompress.streamInner rather than by tree-walk arithmetic. PlaintextCache gains memoize_undeltaed, off by default because a linear sweep that reads each artifact once would only duplicate bytes, and opted into by this walk, which revisits.

Output is byte-identical: the wapp store hashes unchanged, and a 189,251 artifact fossil-corpus repack is a fixpoint at 176,143,384 bytes with the same 146,059 re-deltaed.

test 3495/3495, gate, dst 6x1000, e2e 718/718, git-import-e2e, git-import-cli. check-in: a89b63ad05 user: dmestas tags: sub-manifest

pickBase reconstructed each of its five candidates through the delta chain on every call, and the candidate window slides by one artifact per write, so consecutive calls re-read almost the same set. A store-owned memo cuts frame reads 49.7 percent on pack.

The memo is a LogStore field rather than a caller-owned cache threaded through pickBase and putArtifact. A memo keyed by frame offset goes stale when rollbackArtifactBatch truncates and later writes reuse those offsets, and only the store sees that happen -- a caller-owned cache would hand callers an obligation they cannot discharge. It also changes no signature.

assertBaseMemoBelowEnd enforces the invariant at all three post-open truncation sites. Removing the clear from rollbackArtifactBatch makes it fire from a general import-path test, not only its own regression test. check-in: c075a01cfa user: dmestas tags: sub-manifest

wapp import, user CPU: safe 1076s -> 540s, fast 95s -> 53s. Byte-identical output; the 14 pinned vectors are unchanged.

std.AutoHashMap(u64,u32) + Wyhash is gone. The window is now a comptime 16-byte array read through readInt, and the table is open-addressed with capacity from a bounded counter, so it is never more than half full -- that is the probe's termination argument, asserted rather than assumed.

Most of the safe-side win is not bounds checks. Wyhash.init declares a 48-byte buf as undefined, which runtime_safety poison-fills on every call, and the encoder hashed twice per position -- once for the window, once again inside the map on the resulting key. Removing the map removed both.

The probe's explicit bound sits inside `if (std.debug.runtime_safety)`: present in ReleaseSafe where it can trap, compiled out of ReleaseFast. Written unconditionally it cost 12.2% of the fast import; gated it costs nothing in either mode. check-in: bfb63af209 user: dmestas tags: sub-manifest

zit serve on the imported destination, zit clone over a real socket, then the clone compared to Git commit by commit out of its own store. Proves the packed generation is a usable repository, not merely a readable file.

The mirror fixture gains five validated pairs and an orphan look-alike, so gate 3 is exercised in both directions -- the look-alike's bytes must survive. Author and committer instants are now distinct: git's default made them equal, which left the --attribution author comparison inert, and a regression that always wrote the committer instant would have shipped green. storeHolds errors on an undecodable name instead of answering "absent" to a question it cannot ask; twenty drop-set assertions rested on that. check-in: 4a7c197abd user: dmestas tags: sub-manifest

verify_store, verify_map and after_rename_no_replace had no test; five of the eight legacy aliases were inert selectors that PUBLISH instead of failing. The boundaries are now partitioned into named classes and a gate asserts the partition covers the enum exactly, so a new boundary cannot ship an undriven fault arm.

Cancellation and child failure are exercised while generation 1 is open, with the window proven from the export stream rather than asserted in a comment. Every case asserts launched, reaped, no destination, and a fail-closed no-staging-sibling. Post-rename is inverted: the destination must survive and hold exactly the publish manifest. check-in: 67269dd361 user: dmestas tags: sub-manifest

append(gpa, try dupe(...)) evaluates the dupe first, so a growth failure inside append strands it outside the errdefer. Split into dupe, errdefer, append. checkAllAllocationFailures over mirrorDropNames, with a non-empty drop set asserted so the probe cannot pass on a run that allocated nothing. check-in: ab9a72bd10 user: dmestas tags: sub-manifest

verifyFrozen checked headers and existence -- it opened the map without reading a byte. The ordered pipeline now proves the published generation is the raw one minus exactly the candidate-only drop set: both stores reconstructed and rehashed, name sets differenced, every commit resolved and its tree flattened, tree and provenance digests recomputed, parents and attribution compared, refs and tags resolved, and the repository view derived from scratch. Each mismatch returns its own error and blocks the rename.

Eleven corruption fixtures, one per invariant, and each clause is mutation-proven: disabling it reddens its own fixture and nothing else. A clause no fixture catches is a clause that does not exist.

The ledger's strings borrow the importer, which the freeze destroys before verification runs, so the pipeline captures what it needs before the freeze rather than reading dangling memory.

Real corpus unchanged by the added checks: 7,010,160 bytes published, 0 faults, fixpoint byte-identical.

test 3484/3484, gate, e2e 716/716, git-import-e2e 11/11, git-import-cli 247/247, dst 6x1000. check-in: 60d9edab68 user: dmestas tags: sub-manifest

Translation writes a raw generation, packing writes a second through the classifier's candidate-only drop set, verification reopens both, and only the packed one is published. Stages are one-way and publish rejects every earlier state.

Measured end to end on the wapp mirror at k=5: a fresh import publishes 7.0 MB where the unpacked baseline was 63 MB. zit verify on the result reports 944 artifacts and 0 faults, a second pack is a no-op, and a second import into the same destination refuses with DEST byte-identical.

The catastrophic mode this task introduces -- the 66 MB raw generation riding the whole-directory rename into DEST -- was guarded only by statement order, since .verified is both the pre- and post-discard stage. Two guards now: publish refuses while a raw generation is present, and a staging manifest requires exactly {.zit, .zit-checkout, .zit-import-map.jsonl, refs} immediately before the sync.

Also fixes a use-after-free introduced with the S-tree hints: the partition-key map borrowed an S card's filename, which every frame below the root frees on return. The key is owned now, pinned by a pack-level test that segfaults without it.

test 3471/3471, gate, e2e 716/716, git-import-e2e 11/11, git-import-cli 234/234, dst 6x1000. check-in: 8530aa1440 user: dmestas tags: sub-manifest

deriveCandidateHints read only a check-in's F run, and an S-bearing check-in keeps its files in sub-manifests -- so a tree-shaped repository got no same-path hints at all and every blob fell back to the chronological window. Pack was almost inert on exactly the shape sub-manifests produce.

Measured on the wapp mirror imported at k=5: 66.0 MB packs to 63.9 MB before (508 re-deltaed), 7.2 MB after (1028). 89% instead of 3%. zit verify on the result: 1518 artifacts, 0 faults. Fixpoint holds -- a second pack is byte-identical.

Two lanes, per design §6: blobs take earlier same-path versions nearest-first from the shared map, unchanged in shape and now fed by the full recursive enumeration; S nodes take the PRIMARY PARENT's node under an exactly equal canonical partition key, which needs both trees walked and cannot be an offset-nearest pool -- that is the heuristic the design forbids. Check-in lineage is untouched.

Recursion is bounded by max_tree_depth; a seen-set collapses diamonds and rejects cycles. A referenced S node that will not resolve is corruption and aborts, as the design requires.

test 3461/3461, gate, e2e 716/716, git-import-e2e 11/11, dst 6x1000. check-in: 6e2fc456f8 user: dmestas tags: sub-manifest

A Fossil->Git mirror carries root manifest/manifest.uuid/manifest.tags that are not project content: wapp has 252 distinct manifest blobs across 252 commits while its native history tracks no such file. The classifier proves per commit that those bytes are reconstructible from what the import already holds, so packing can drop them.

Fail-closed and cryptographic: stripped-manifest hash equals the uuid, F cards equal the tree, P cards equal validated parents, D card equals the committer instant. A real top-level file named manifest cannot satisfy that by chance, and any inconclusive signal retains every present candidate -- never a strict subset.

Three corrections to the design, all established against the corpus: presence is per-repository (fossil's manifest setting emits each of the three independently), gate 6 must exclude every present candidate since F cards list none of them, and gate 3 must strip the sterilizer line the export appends or the uuid never matches. update.zig's manifest_paths and sterilizer are now public rather than duplicated -- it already had all three names and the exact trailer.

Measured on the pinned wapp corpus: 252/252 commits validate, drop set 512 = 252 manifest + 252 manifest.uuid + 8 distinct manifest.tags.

test 3461/3461, gate, e2e 716/716, git-import-e2e 11/11, dst 6x1000. check-in: d189b0da1d user: dmestas tags: sub-manifest

applyDirect's >= skip means a tag on a separate control artifact, rather than on the check-in manifest, depends on crosslink order; names emitBranchControls as the dependent. Also corrects branchValueOf's doc, left stale by c8c7c2696e. check-in: 941ddcb981 user: dmestas tags: sub-manifest

fast-export emits each commit once, under whichever ref reached it first, so that header was provenance and not branch. A single-branch repository imported with ZERO branches; multi-head repositories were scrambled in proportion to merge activity, which at sqlite scale read as 7,421 leaves on master under repo-state §7.2's same-branch leaf test. Issue #26.

labelBranches claims the primary's first-parent chain, then each head shallowest tip first, then inherits; emitBranchControls writes one control per segment root. Two things the plan did not anticipate: objects are folded to check-ins by hash first, because two git commits can translate to one byte-identical check-in and §5.2 allows one branch tag per rid; and controls are emitted deepest-first, because §5.3's timestamp skip would otherwise let an ancestor's propagation win over a root whose commit shares its second.

source_ref is deleted rather than demoted -- after the change it had no readers. makeBranchControl now composes the §6.2 pair through artifact.composeCommitTags instead of a lone *branch card.

Content was never affected: artifacts, parent links and tip content were always correct. Only derived branch state was wrong, which is why a full benchmark ran clean through it. Repairing an existing imported store means re-importing; check-in hashes survive, since branch state lives in separate controls.

Two pre-existing expectations were rewritten, deliberately: both demanded two active branch tags on one rid, which §5.2 forbids, and were unsatisfiable rather than merely stale.

Verified: fmt, test 3281/3281, gate, e2e 716/716, git-import-e2e 11/11, dst 6x1000. Shapes A/B/C match git branch exactly, 11/11 tip parity. check-in: 4d7a22b93c user: dmestas tags: sub-manifest

repo-state §6.1 requires reading the configured value and falling back to trunk only when main-branch is unset -- "MUST NOT assume a fixed string in the presence of an explicit setting". branchValueOf, which backs the §7.2 same-branch leaf test, used the literal.

It hid because all three uses are comparisons: when both sides fall back identically the result is unchanged. It bites when one check-in is tagged and another is not, which is the imported-repository shape -- displayed as one branch by branchOf, leaf-tested as another by branchValueOf.

The name is now a field on RepoState, set through one RepoView setter so the two readers cannot drift, with branchOf asserting its argument agrees. That assert immediately found a second source: update.zig re-read the config itself and passed a value the state never saw. It also treats an empty stored name as unset, as the view already did. check-in: c8c7c2696e user: dmestas tags: sub-manifest

An S-bearing manifest carries its files in sub-manifests, not in its own F run, so reading bare .f found none of them and every tracked path was classified `added` -- identical output whether or not anything had changed. Issue #25.

CheckinBaseline and DiffBaselineResolver now flatten first, the same step commit already performs. §5.1's "downstream MUST NOT distinguish" is exactly this: a raw .f read is how a reader opts out of the derivation seam.

Submanifest-specific, confirmed both ways: a --no-submanifests import was already clean. Not import-specific though -- any tree-shaped checkout hits it, so a clone from a tree-emitting peer would too.

Regression test asserts clean status and empty diff after materializing a tree check-in, and that a real edit still reports `edited`. check-in: 1e8c8a198d user: dmestas tags: sub-manifest

hexToDigest required exactly digest_len*2 hex characters. zit init writes the sha3-256 policy, so that looked like the only case -- but import-git opens its destination under git's sha1, naming every check-in in 40 hex. So commit rejected the parent of every imported repository and reported it corrupt on a store zit verify called clean. Issue #24.

Length now comes from the name, as artifact §7.2 allows both widths. pack.zig carried the same copy and would have dropped the primary-parent hint the same way; cmd/sync.zig's callers move to the slice form.

commitParentCorrupt now names which step failed. Collapsing an undecodable name, an absent parent and real damage into one sentence is why verify and commit could disagree with no way to tell which was right.

Reproduced on 12 commits and 40 files -- not scale-dependent, and not submanifest-specific: it fires before the S-card branch, so flat imports failed identically. Regression test commits over an imported sha1 history; mutation-proved by restoring the width check. check-in: ffea6238e6 user: dmestas tags: sub-manifest

Same source, same path, two imports: check-in manifests byte-identical once F/S/Z/P are stripped, blob provenance identical, P topology re-derived through each import's own map. check-in: 81b01a1157 user: dmestas tags: sub-manifest

Imports with sub-manifests off: flat manifests throughout and the spec-default config frame (k=0, enable off). Flat is §8.1's default shape; it is this verb's analyze-and-pin that is the deviation.

TreeFormat.Shape becomes flat | tree{k, height_max}, so flat-with-a-k and tree-without-one stop being representable and every tree-only site traps on the wrong tag. buildFlat drains the same MergedCursor, so ordering, dedup and rename/copy/delete stay single-sourced. SubmanifestChoice replaces submanifest_k from Request down to UserOptions.

Publication refuses if the store holds a SUBMANIFEST anyway, derived from work done rather than from the flag. Preflight analysis still runs under flat -- skipping it would put a second divergence inside the region the benchmark times.

Closes #23. check-in: 9b96469ae6 user: dmestas tags: sub-manifest

The latch is one-way and means 'an S reference is stored here', not 'trees are enabled'. The import set enable_submanifests and left it false, which stayed hidden only because the §7.2 gate is the disjunction of the two -- a later 'zit submanifest disable' would have left the repository advertising avoid-submanifests while holding S-bearing artifacts.

Derived from emitted_nodes, not from the k pin: an import whose sources all stay flat stores no S reference and must leave the latch false. check-in: 0b6e41e54f user: dmestas tags: sub-manifest

Additive; covers the tagged trailing block and the legacy widths. check-in: 3392ff5a3e user: dmestas tags: sub-manifest

Deletes the duplicate resolver in update.zig. The surviving path decodes S-card hashes case-insensitively per §3.5; the deleted one ran them through lowercase-only validateName and rejected a legal uppercase hash. TreeError codes mapped exhaustively rather than via else. check-in: 4a3061280e user: dmestas tags: sub-manifest

Four blocks of prose attributed to §6.4 a sentence that appears nowhere in the draft; replaced with verbatim draft text.

§6.4 requires the relative margin m to be documented, which a source comment does not discharge. `zit commit --help` now states the 15% threshold, with a test that renders drift_margin_pct and requires the help to contain it -- the prose and the constant cannot drift apart.

The config frame's trailing block carries a shape tag, so an extension a reader cannot name costs one re-scan instead of an unreadable config (login, remote and project fields included). Legacy widths still parse.

Names the §7.2 avoid-submanifests predicate: the pragma keys on the setting alone, not the armed disjunction, and that divergence is now visible at the call site.

These four land together because D's deletion and E's addition are interleaved in one region of repository.zig with no hunk boundary between them. check-in: 999efb0fd9 user: dmestas tags: sub-manifest

The auto-retune erratum was accepted and is in the draft body; these comments still said it was submitted and unapplied. Comments and one test name only. check-in: a2eb78b7be user: dmestas tags: sub-manifest

Bench now builds ReleaseSafe, which is the mode the O(store) asserts fired in, so without this the clone arm hits them at width. check-in: c1f44ba0ed user: dmestas tags: sub-manifest

The emitted `optimize` field was the string "ReleaseFast", hardcoded into the JSON format string rather than derived from anything. It reported ReleaseFast for every record the benchmark has ever written, whatever the executable was actually built as - so the one field whose job is to say which binary produced a measurement could not be wrong in a detectable way, only in a silent one.

Now `@tagName(builtin.mode)`, alongside the os/arch tags beside it that were already derived that way.

Found by switching bench to ReleaseSafe and watching `executable_sha256` change while `optimize` did not. check-in: be18ff3902 user: dmestas tags: sub-manifest

`--release` resolves to ReleaseSafe here as a correctness decision: src/ carries over a thousand asserts that turn a detected inconsistency into a crash instead of a durable write, and ReleaseFast compiles every one of them away. The bench executables opted out of that on the grounds that "a benchmark must measure optimized code".

Optimized, yes - but it has to be the code users run, or the number describes a binary nobody executes. This matters more than a uniform constant would suggest, because the cost is not uniform: the same rationale block records `zit pack` at 599 ms fast against 15.2 s safe on one store. A measurement taken in the mode that strips the asserts cannot be compared against, or used to make a claim about, the shipped artifact.

`bench_zit` is the sharpest case - the serve benchmark spawns it as the real `zit` binary, so it stood in for production while being built unlike it. check-in: 30b7d5e766 user: dmestas tags: sub-manifest

crashCheckOnce repairs by truncating below the earliest injected bitflip and reopening. When the truncation empties the log, that reopen writes a fresh header - and the injected short-write fault could land on it, so open returned Storage, the sweep mapped it to Io, and the run failed. Nothing there was a statement about the store: the simulator faulted its own recovery.

The repair and its reopen now run against the healthy disk phase 2 heals to, the all-zero FaultCfg. Faults come back before the invariant check, which is workload again.

Pre-existing: trunk fails seed 201 with the same Io. check-in: 9e3107e570 user: dmestas tags: sub-manifest

The generator's own note says a real check-in DAG is acyclic by construction - a manifest's hash covers its P-card - while the phantom-parent flavor lets a hash be named before its record exists, so a later pick can close a loop and must be checked. The check only ever walked primary_parent_of, and the merge parent was drawn by pickInsertedIndex with the comment "no cycle risk".

That holds for the primary chain alone. Section 3.3 ancestry, which the B1 tie-break walks, follows merge parents too: a merge edge into an already inserted check-in, paired with a phantom primary edge out of it, closes a loop the guard never looked at. ancestryMaximal then finds every candidate dominated and no maximal element, and panics - seed 192 built exactly that, 9 to 14 to 13 and back.

parent_indices records every parent index a record names, wouldCloseCycle walks that graph visited-once, and the merge draw is checked like the primary one. A losing draw degrades to the single-parent record rather than drawing again, so the seeded stream shifts as little as it can.

Pre-existing: trunk fails seed 192 identically, and gen.zig is byte-identical there. Only the default six seeds hid it. check-in: 992ff19a23 user: dmestas tags: sub-manifest

A name can reach the same check-in through two index entries, so one selection offers the same rid twice. MtimeMaxPick.offer keeps the first as `best` and appends the second to `tied`; winner then appends `best` back, so `tied` holds that rid twice.

Ancestor sets are self-inclusive, so each copy found itself in the other's set. Dominance skipped only the same INDEX, so both copies were dominated, every candidate was eliminated, and the maximal element a finite DAG must have came up null - a panic on `best.?`, reached through resolveName.

Dominance is about being a different check-in, not sitting at a different offset. The new test drives offer/winner directly, and also pins that a duplicate cannot shadow a real descendant. check-in: d3f58d2875 user: dmestas tags: sub-manifest

Trunk's Slice 1/2 made both open paths header-only, so a rotted payload now opens clean and surfaces at content() instead of at open. The storage sweep still assumed open was the tripwire, so two of its own claims were wrong once the merge combined them with this branch's store:

- crashCheckOnce repaired only "if the open itself reports corruption", a precondition Slice 2 removed. Injected rot reached the invariant check and read as data loss. NeedsRepair, private to the harness, now carries "refused AND genuinely damaged" to the repair that already existed.

- rotWithinFrame spelled the per-frame overhead as 22, the header alone, so a frame's own trailing frame_crc32 counted as inert pad. A bitflip in those four bytes makes a reader recompute a correct checksum, compare it to a corrupted stored one, and refuse the frame; the sweep called that data loss. logstore now exports frame_fixed_overhead and the sweep derives it instead of respelling it.

Both are trunk defects, not merge damage: trunk alone fails dst at seed 8. The merge only shifted the draws far enough to reach the first one at seed 3, inside the default gate's range.

cli.zig's verb enum was the single textual conflict: a pure union of trunk's verify and this branch's submanifest and import-git.

fmt, test, gate, dst green. check-in: 584314dcb7 user: dmestas tags: sub-manifest

zit now imports sqlite at defaults: 36,416 commits, 2,109 refs, 44m35s, no flags. Two independent walls stood in front of that.

Refs (issue 21). One file per ref made a ref's name a path, so the filesystem got a vote on which repositories are representable. sqlite has refs/remotes/origin/EQP-improvements and .../eqp-improvements at different commits -- legal, distinct git refs -- and on APFS the second landed on the first as a bare PathAlreadyExists, raised during publication after the whole import had run. Case was only the reachable instance; path length, reserved device names and each platform's own Unicode fold table are the same bug in other clothes.

They are one sorted file now, "refs", whose lines pair a canonical name with its target. Nothing reads it back -- "refs" is a reserved root the working-tree scan skips, and branches resolve through repostate's sym- tags -- so it is interop output and its job is to be legible. Sorting makes it byte-identical for a given ref set, so two imports of one repository can be compared.

This supersedes the ASCII-fold refusal from the previous commit. That was the right interim while a ref was a filename; with one file the collision is representable, so refusing it would be gratuitous. What survives is the genuine collision -- distinct sources normalizing to a byte-identical canonical name, refs/heads/./release against refs/heads/release -- which no layout can hold, and which now names both sources instead of only failing.

Budget (issue 20). The 4 GiB payload default refused sqlite's 12.2 GiB stream, and could not simply be raised: fileCommandCeiling divided the same number, so a larger allowance for a whole history was also a larger allowance for one commit's retained command list. Those bound different things and only one is memory -- payload streams and is released per event (sqlite's import held 21 MiB), while a commit's command list is held whole. The arena has max_command_arena_bytes of its own now and keeps the old 4 GiB; the cumulative total goes to 1 TiB.

The file-command ceiling is therefore structural: about 104 million commands in one commit, and no flag raises it. Its diagnostic says so rather than pointing at a knob that no longer moves it. The help text claimed the payload flag fixed both; it only ever fixed one. check-in: 8ae7041c2c user: dmestas tags: sub-manifest

sqlite carries refs/remotes/origin/EQP-improvements and .../eqp-improvements pointing at different commits. Both are legal, distinct git refs. Zit stores a ref by creating a file at its canonical path, so on APFS the second one landed on the first: bare PathAlreadyExists, raised by writeRefs during publication, after 49 minutes of completed import. No ref name, no cause.

The refusal already existed. finalizeRefs rejects two source refs that resolve to one canonical leaf -- 'refs/heads/./release' against 'refs/heads/release' is a tested case. It just compared bytes, and a filesystem does not. Keying the canonical index by an ASCII fold makes the existing rule see the collision it was written to catch, uniformly, instead of leaving it to whichever disk the import ran against.

ASCII rather than Unicode is deliberate and documented at the fold: APFS, HFS+ and NTFS each fold a different frozen table, so following them would make the importable set depend on the volume. A-Z is what they agree on.

The error now names both source refs, both canonical spellings, and whether case alone separates them -- a normalization clash and a case clash want different fixes from the operator. Retained on the importer and formatted by the CLI, the same split FileCommandOverrun uses.

This refuses on Linux what Linux tolerated, which is the point: the repository means the same thing everywhere or it is not portable. Making sqlite importable rather than merely diagnosable is a layout question -- one file per ref cannot represent it -- and stays open. check-in: f0331d2f5a user: dmestas tags: sub-manifest

ExportPayloadLimitExceeded was a bare error name. Importing sqlite cost 17 minutes to learn a single integer, and the operator still had to guess it: the stream is 12.2 GiB against a 4 GiB default.

One error carried two unrelated bounds. max_total_data is the operator's --max-export-payload-bytes; max_data is a fixed 1 GiB per-payload ceiling no flag reaches. Reporting the wrong one sends an operator to a knob that cannot help, so PayloadOverrun records which bound refused, and the single -payload arm says plainly that no option raises it.

The cumulative arm refuses to name a remedy value. Parsing stops at the overrun, so the observed total is a lower bound; 'above 9' would name a number that fails again. It reports the bound and says the full payload is larger, which is the honest shape of what the parser knows.

Also corrects the budget's own doc comment, which claimed four GiB was 'the working set an import may claim on a commodity machine'. Measured resident size importing sqlite's 12.2 GiB stream: 21 MiB. Payload is streamed and released per event. What the cumulative half actually bounds is total history, so the default over-provisions memory by three orders of magnitude while refusing a mainstream repository. check-in: ee703cde16 user: dmestas tags: sub-manifest

Trunk since d186a44e brought the repository shun table and its whole- branch verification fixes, the release-assert guard, the `test`/`gate` lane split, the structurally bounded logstore priority window, and the vendored crc32. Eight files conflicted. The decisions worth naming:

Lanes. `git-import-e2e` was a compile-only proof hanging off `test`; that is exactly the cost trunk moved to `gate`, so it moves with `e2e` and `smoke`. `import-git-bench` stays on `test` because it registers a real unit-test run rather than a compile proof. `declared_steps` is the union plus `gate`.

Sync ingest. Both sides rewrote it: trunk added the §7.7 receive-side shun admission, this branch added §7.1 phantom minting for a pushed tree's missing sub-manifests. They compose in one direction — admission is asked first, so a card §7.7 refuses is never stored and mints no phantoms either. The S references of an artifact this repository declines to hold are not names it should then go begging for.

Client construction. Trunk factored the three verbs' `Client` literal into `syncClient` and threaded `RepoAdmission`; this branch had widened `project_code` into `RepoSyncIdentity`. The helper now carries both, which is what trunk's own comment asked for: three copies of that literal is how a repository filter comes to be wired into two verbs and not the third.

Per-round server refresh. Trunk made the snapshot mandatory — `error.ServerStorage` rather than serving with no shun table — where this branch read its four sub-manifest fields through fail-open defaults. Trunk's reading wins for all six fields: serving a stale default is the failure, not the fallback.

Two guards trunk owns caught real defects in this branch's code, which is the whole point of them:

The total switch in `shunConsequence` refused to compile against the `submanifest` artifact type, exactly as its comment promised. It has an arm now.

The release-assert guard found nine asserts on untrusted input across six functions. Two were reachable faults, not style: `lookupTree` asserted §4.4 on a sub-manifest a peer supplies, so a malformed node panicked a ReleaseSafe reader and was walked anyway under `--release=fast`. It returns `SubUnresolvable` now, which §5.2 already defines as "absent, or is not a SUBMANIFEST". The empty-S-run assert became an early `null`. The two sub-manifest writers refuse an empty group through a new `SubmanifestWriteError` rather than asserting; the builder's internal path keeps its allocation-only surface through `serializeLeafCore`, since the partitioner is what establishes non- emptiness there. The rest were tautologies or restatements of invariants the card parser already enforces, and now say so in a comment instead of carrying a panic.

Verified on the merged tree: `gate`, `e2e`, `dst` (6 seeds x 1000 steps), `smoke` (S1-S10, including trunk's new shun interop), and `git- import-e2e` all green, `zig fmt --check` clean. Beyond the suite, the importer was run for real: the empty-email repro imports and its check- in carries `U mohye77`, and marktext imports end to end with the analyzed pin at k=5. check-in: 562cf5be74 user: dmestas tags: sub-manifest

`zit import-git SOURCE DEST` reads a Git repository through `git fast- export` and builds a zit repository from it. The destination appears only after the export, artifact creation and publication have all completed, so a failure anywhere leaves no half-published repository behind; this is the first caller of the logstore artifact batch added earlier on this branch, which is what makes a check-in's blobs arrive as a unit.

Mode 160000 now parses as a gitlink instead of error.UnsupportedMode. That completes the split begun in the previous check-in: reporting what the stream said is the parser's job, and deciding that zit has no submodule concept is the importer's. The refusal still happens, but it now explains itself instead of failing as a parse error on input git legitimately emits.

Signed commits are refused by default rather than imported with the signature silently dropped. Fossil has no signature card, so importing one is a loss the operator should choose; --drop-signatures accepts it explicitly.

The sub-manifest parameter is analyzed, not guessed. Preflight measures the source's own name set and pins its measured optimum, and --submanifest-k overrides that only to match another repository's existing pin, which is what lets two repositories share sub-manifests by content address. On a 100,000 file corpus the analyzed value cuts the objective by roughly 3.4x against the fixed default it replaces.

Correctness is checked differentially against real Git rather than against fixtures we wrote: every materialized tree is compared to `git ls-tree` for the corresponding commit, refs and annotated tags are compared to the source, and both comparators now assert that a gitlink is absent rather than merely tolerating it, so the omission is a contract instead of an accident.

The rest is wiring. extras learns the import provenance names, root.zig exports the importer modules across the executable and library boundary, tidy and the CI step guard learn the new test roots and build steps, and the e2e help check learns that a verb spelled import_git in Zig is import-git on the command line. check-in: dc0aa2f858 user: dmestas tags: sub-manifest

The incremental tree editor diverged from buildTree on realistic filename distributions. OldTreeCursor.appendPrefixAtHeight read the retained native-prefix list without consuming it, so one descriptor could be spliced into two different output levels: buildUnpublished placed the height-1 prefixes into the leaves, and mergeRootPrefixes then prepended the identical descriptor again before finishRootLevel. The emitted interior node carried the same S key twice, which fails artifact.parse's ascending-S guard, so an import aborted partway through a real history with a SyntaxError raised by our own grammar.

finishCheckpoint already guarded its own splice with a height test. That guard was an ad-hoc workaround for this same double-append, and mergeRootPrefixes never had one. The splice now consumes what it emits, so a retained descriptor lands in exactly one level and the guard becomes unnecessary rather than duplicated.

Found by importing marktext, not by the suite. Every name corpus here was uniform, and a duplicated prefix descriptor requires names that share prefixes at differing depths, so 55 tests could pass over a defect that breaks real repositories. The new seeded differential sweep generates four corpora of 136 to 240 paths with mixed depth, shared directories, dotfiles and 1 to 43 byte basenames, over every supported k with twelve batches each, and asserts it reached a tree at least three levels deep so it cannot quietly degrade into testing flat roots. It diverges in 20 of 452 batches against the unfixed editor, every divergence unparseable.

The decoded-root validation memo lands alongside it: repeated lookupTree validation against an unchanged root is answered once instead of rebuilt, which takes this file's own 56 tests from 1701 to 450 seconds over identical assertions. check-in: 4a95e436af user: dmestas tags: sub-manifest

Limits.max_file_commands_per_commit was a standalone number, and the caller pinned it to the width of our own benchmark corpus. That is a fixture size promoted to policy: no headroom, no way to raise it without a rebuild, and it refused legitimate histories. `git fast-export` lists an entire tree for a root commit and for any commit whose first parent was not exported, so the ceiling is reached by ordinary repositories whose initial commit lands a whole codebase at once.

The bound itself is necessary. admitData is the parser's only byte admission and it sees the byte count in a `data` header alone; every file command additionally allocates a list slot and a duplicated path into the per- commit arena with nothing charging for them, so the command loop is otherwise unbounded against a subprocess pipe.

It is now computed: fileCommandCeiling(limits) = max_total_data divided by min_retained_bytes_per_file_command, where that constant is @sizeOf(FileCommand) plus one -- a command's list slot plus the shortest path it can carry, the empty path being rejected. Nothing smaller is representable, which is what makes it safe to divide a byte budget by. The ceiling restates the arena authorization the caller already granted, in commands rather than bytes, and moves when that authorization moves.

Refusal retains the observed count, the ceiling, and the budget that produced it, so the diagnostic names all three rather than only its own error. The remaining limits gain distinct names -- Data, Event, FileCommand, Line, Merge and Path -- in place of one LimitExceeded that could not say which bound was hit. Exact-count data may carry an optional trailing LF, so only the first following empty line is ambiguous with the commit terminator; readDataExact and readCommitData split that decision instead of guessing. check-in: 85d510d361 user: dmestas tags: sub-manifest

Names beginning with `sym-` are ambiguous in this repository. A user tag written `sym-x` is stored under the canonical spelling `sym-sym-x`, so resolving `sym-x` correctly looks for `sym-sym-x`. An imported repository, though, can carry a tag whose stored canonical spelling already begins with `sym-`, and nothing could address it: the ordinary lookup rewrote the name and missed the artifact that was actually there.

Resolution now attempts the ordinary user-tag path first and consults the stored spelling only when that fails. The precedence is deliberate. A user tag continues to win over an imported canonical name, so no existing repository changes meaning, and the fallback is reachable only where the ordinary answer does not exist at all. check-in: fc9bcb81bd user: dmestas tags: sub-manifest

Callers publishing several artifacts together had only putArtifact, one at a time, so a failure partway left the store holding a prefix of the set with nothing naming what was missing. Each caller that cared wrote its own compensation, or quietly did not.

putArtifactBatch takes the whole set and appends it under a single transaction: either every item is durable or the store is byte-identical to its prior state. ArtifactBatchItem carries the hash, the bytes, and an optional delta base, so a batch may mix full and delta-encoded members without the caller resolving storage order first.

Nothing calls it yet. The importer, which needs a check-in's blobs to arrive as a unit or not at all, is the first caller and lands later on this branch. check-in: 7cf522fa5b user: dmestas tags: sub-manifest

Two layers decided independently what a reserved path is. update carried its own validateWorktreePath — an absolute/backslash/dot-dot check plus a hand-rolled case-insensitive comparison against the .zit family — while extras carried isControlName, which tested only the first /-delimited component of a path. They agreed by coincidence rather than construction, and the extras form admitted `.zit-managed/entries` because it asked about the component instead of the root.

control_paths now owns both questions. validateWorktreePath and isReservedRoot are the single definitions; update delegates to the first and extras to the second. isControlName takes the full root-relative key that add and move already hand it, so a path rooted in a control entry is refused at any depth rather than only at depth zero. firstComponent goes with its last caller.

The update test that pinned the old list is now a case table naming what is reserved and what merely resembles it, including the case-insensitive spellings the previous ad-hoc comparison happened to get right. check-in: 581db6e5ba user: dmestas tags: sub-manifest

A check-in's F card can name a symlink (§4.7.6-m5), but update refused one outright with error.SymlinkUnsupported, so any history carrying such a card could not be checked out, reverted, or updated to. The refusal was cheap to write and wrong: the reader already represents the entry, and only the worktree side lacked a branch for it.

materializeEntry and writePayload now create the link from the card's target bytes instead of writing a regular file containing them, and isCurrentState compares against .sym_link and the recorded target rather than the size and mode of a file that is not there. The managed-set walks accept .sym_link beside .file, so a materialized link reads as a tracked entry instead of an intruder the admission check rejects.

`zit revert` on a symlink card previously exited 1 naming the refusal. It now restores the entry, and the test reads the link back with readLink, which distinguishes native materialization from a regular file that merely holds the target's bytes. check-in: bb96853a0f user: dmestas tags: sub-manifest

The e2e root is one test binary holding every scenario, and its runner took no filter, so narrowing to a single scenario meant editing build.zig or paying for the whole root. `zig build e2e -- --test-filter NAME` now compiles the filter into the test artifact, accepting both the two-argument and the `--test-filter=NAME` spellings and passing nothing through when the arguments do not name a filter. check-in: cd70f19286 user: dmestas tags: sub-manifest

Trunk carried the pre-errata draft. The corrected text -- the SS3.3 shun-table definition and the SS7.7 receive-side shun admission rule -- was only ever committed on the sub-manifest branch, so any branch cut from trunk was building against superseded normative text. This is the file the errata records name as applied (sha256 068d7672...). check-in: d186a44ea3 user: dmestas tags: trunk

.agent-config/ collects per-session tooling traces at the repo root. Being dotfiles they never appeared in a bare 'fossil extras', so they were held out of commits by fossil's default alone rather than by any rule -- one 'addremove --dotfiles' would land them permanently. CLAUDE.local.md is checkout-local operator config. Both are globbed now and pinned by packaging_test. check-in: a27cf4b4a9 user: dmestas tags: trunk

std.hash.Crc32 is byte-at-a-time over one table (~350 MB/s here) and the open-time scan CRCs every payload byte, so it was ~93% of a cold open. Slicing-by-8 in its place: a 416 MB store opens in 292ms against 1.22s, warm reads run 3.3x, the write path 1.5x. Bit-identical by construction, so no format changes and no repository is affected. check-in: 602c2459f2 user: dmestas tags: trunk

Both consumers live in their own build roots and cannot reach crc32.zig by path, so the export is forced. Adds the differential fuzz lane against std — the only coverage that can see a self-consistent but numerically shifted implementation, which would orphan every existing .zit file.

The implementation brief prohibited this export, citing repocache as the precedent for unexported leaf modules. That precedent does not apply: repocache has no out-of-module consumer. storestat and repoimage, which do, are both exported. Overridden deliberately. closed check-in: c57c16c5cd user: dmestas tags: logstore-index

The other three on-disk formats defined in terms of this checksum: the cache image self-checksum, the bisect session trailer, the undo journal trailer. Same swap, same values. check-in: 55d6e7c7bb user: dmestas tags: logstore-index

The four frame checksums plus the file header. The open-time scan CRCs every payload byte, so this is ~93% of a cold open: 416 MB goes 1.22s to 292ms. No format change — the values are identical by construction. check-in: dd71dd3006 user: dmestas tags: logstore-index

Bit-identical to std.hash.Crc32, which every on-disk checksum in the store is already defined in terms of. std's is byte-at-a-time over one table (~350 MB/s here); this folds eight bytes per step. Nothing uses it yet. check-in: aef8f539b5 user: dmestas tags: logstore-index

Emission is off by default and turning it on is an explicit act that must name the partition parameter: there is no fallback value, enabling without one is refused, and a pinned value outside the supported range emits flat manifests and says so rather than substituting a parameter of its own. Turning it off returns the next commit to flat. The analysis pass reports what the parameter would cost over a name set; it pins nothing.

A server whose setting is off now sends the avoid-submanifests pragma, which the receiving side already latched. It leads the pragma run in both the ordinary and the clone reply, so six tests that read a reply's first card were updated to account for it; each still asserts the same content it did before, behind the new card. Fossil 2.28 tolerates the pragma and the interoperability suite is unchanged.

The parameter is repository-local. The specification requires it to propagate to clones, since sharing by content address holds only while every writer partitions identically; that shortfall is recorded rather than implied away. check-in: 92a40960d1 user: dmestas tags: sub-manifest

An absent S reference now phantomizes like any other missing reference, its dependent is deferred and retried when the artifact arrives, and the sweep excludes sub-manifests so it still terminates. Phantoms survive a restart through a rebuild-on-open invariant resting on a durable seen-submanifest setting rather than on records the log never wrote. The transfer gate decides by artifact shape on all three send paths, so a leaf is refused wherever a tree manifest is, and the memo returns its own copy rather than a view into pages the store may free.

The gate remembers what it withheld from a peer, which is what lets an exchange with a peer that refuses sub-manifests terminate at all. That record is now cleared only when the peer's announced version actually changes: a peer that re-announces the same version in every reply was clearing it every round, so the same artifacts were offered and refused forever.

One requirement is not implemented: a server whose setting is off must send the avoid-submanifests pragma, and emitting it changes the card sequence every clone-reply test walks. That is a protocol-shape change worth its own review rather than a rider here; the pragma is parsed and latched on receipt already. check-in: 8446adcd98 user: dmestas tags: sub-manifest

Sub-manifests have no reference implementation to check against, so the corpus is the oracle and had to be built rather than assumed. Adds the eleven vectors Appendix A.5 requires that were missing, among them the three-level tree whose absence is why the step-5 cursor contradiction survived into an implementation: no published artifact carried an interior node, so nothing exercised that path.

Every authored vector is generated by the implementation and then rechecked along a path that shares no code with it, and each derived name is compared against the constant the test pins, so no hash in the corpus is hand-written. Two fuzz targets cover S-card classification and the tree walk.

One A.5 bullet is not satisfiable as written: the re-tune vector asks for two parameters over the A.1 name list that yield trees sharing no sub-manifest, but that list's single qualifying boundary falls in the same place for the first three values and is suppressed above them. No vector was invented for it; an erratum is drafted and a test pins the finding so it stays visible. check-in: 72bba34ada user: dmestas tags: sub-manifest

Regenerated from the RFC suite after the erratum submanifests-00-s5.1-step5-cursor-advance was applied. Step 5 of the tree walk now advances the referencing frame's cursor before pushing the interior frame; as published it omitted that, so returning from an interior descent re-read the same S card and failed the strictly-greater-than-LAST check, leaving every tree with an interior node unwalkable. The walk already implements the corrected reading. Also picks up an MD5 citation and reflowing in unrelated paragraphs. check-in: f2973c41c3 user: dmestas tags: sub-manifest

The merge fires when the two groups together hold at most MAX, and no test separates that from a strict comparison. Reaching the distinguishing case needs a natural boundary at exactly MAX minus the tail, which hash-driven boundaries do not produce in generated name sets; a search over two parameter values and nine hundred starts found no instance. Pinning it would mean choosing filenames by their leading zero bits rather than generating them. The note records the reasoning at the site so the gap is visible to the next reader. check-in: 9897845809 user: dmestas tags: sub-manifest

The S hash is validated case-insensitively and kept raw, so a parsed card carries whatever spelling it was given. The writer asserted the narrower lower-case spelling, so re-emitting a card the parser had just accepted tripped the assertion. It now asserts the parser's own predicate.

Reachable only once something emits, which nothing does yet; found by round-tripping an upper-case hash through create. check-in: 6e64c8976c user: dmestas tags: sub-manifest

Implements draft-fossil-submanifests-00 section 6: zbits over the decoded filename, the group scan with its MIN/MAX member bounds, and the final-group merge. One scan serves height 0 and every height above it, so the merge applies uniformly rather than through two code paths that can drift; appendix A.4 is the vector that fails when they do.

Construction emits filenames through a minimal encoder scoped to its own output only. Existing emission keeps the escaping it has always used, so no check-in already written changes its bytes. R cards are emitted for tree manifests alone.

Nothing emits yet: the construction is wired behind a disabled flag, leaves stored before the nodes that reference them. The gate that turns it on is a later change. Tests pin k = 3, the value the appendix vectors use, and cover byte-identity against A.2, A.3 and A.4, determinism across repeated builds, and a constructed tree walking back to the entries it partitioned. check-in: 9d7992613f user: dmestas tags: sub-manifest

Implements draft-fossil-submanifests-00 sections 5.1 through 5.6: a stack walk over S runs with the section's depth bound, cycle check, strict ascent and first-filename identity, resolving sub-manifests through the existing baseline resolver. A tree that fails any check fails whole, so a partial enumeration is never presented as a file set.

The walk sits in a pre-pass at the facade, leaving enumerate, lookup and their call sites untouched as section 9.3 requires. R-card verification runs over the walked enumeration.

Step 5's cursor advance follows the erratum submanifests-00-s5.1-step5-cursor-advance: as published, step 5 omitted it, and no tree carrying an interior node could be walked. check-in: d1f955aaf3 user: dmestas tags: sub-manifest

Adds S as the twenty-second card letter and SUBMANIFEST as the ninth artifact type, per draft-fossil-submanifests-00 sections 3 and 4. Type determination reassigns after the card scan rather than during it, so no type is committed while cards are still being read (section 9.1). Reading and classification only: no tree walk, no emission.

The draft's seven appendix vectors ship alongside, each named by the SHA3-256 the draft states and checked by a test that recomputes it. check-in: 9b8bad477a user: dmestas tags: sub-manifest

A serving release over 0.1.2. Where 0.1.2 made a large repository cloneable, 0.1.3 makes serving one fast and lean.

A server against a large repository comes up immediately: the listener opens before the repository is read, so a client queues in the accept backlog while the server primes rather than being refused during a startup a large store made long. A push no longer stalls inside its own reply deadline; the snapshot rebuild it triggers now runs between connections. Serving reads the repository through an on-demand log-offset index rather than a resident copy, so peak memory to serve a 67,499-artifact repository fell from about 157 MB to about 32 MB, with clones byte-identical to before.

Also: the end-to-end suite is safe to run concurrently with itself (fresh loopback ports), and the unclassifiable-artifact tolerance rule is now covered by deterministic simulation and an end-to-end scenario. check-in: a469147254 user: dmestas tags: trunk, version-0.1.3

The rule that lets state assembly skip a stored artifact it cannot classify — when the artifact's bytes verify against the name it is stored under — shipped with the fix but was pinned only by two unit tests over hand-built logs, and by a manual clone of the real fossil repository. Neither the simulator nor the end-to-end suite exercised it, so a regression could pass every automated gate.

Deterministic simulation grows an artifact-level generator (src/sim/ assembly_sweep.zig): it emits artifacts whose card sets sit in and around the required-card table — some verifying against their stored name, some stored under a wrong one — and asserts the split directly, that assembly completes iff every unclassifiable artifact verifies, derives no state from one, and stays fatal on a single artifact whose bytes do not match its name. It runs every seed. assembleState is made public for it to drive; its body is unchanged.

The e2e suite gains a scenario that clones a served store carrying an anonymous wiki artifact (cards D L W Z, no U — the shape that first surfaced this) and asserts update tolerates it, with the read verbs and assembly agreeing on the same store. Reverting the verify-then-skip arm to an unconditional failure reddens the sweep, the e2e scenario, and both unit tests. check-in: c2851f933e user: dmestas tags: trunk

Priming a serve snapshot copied every artifact's container out of the log into an in-memory store and held them there for the server's lifetime. The log had already been scanned and every frame CRC-checked to build the rid table, so the copy was a second full read whose only product was resident bytes: on a 254k-artifact repository it was two thirds of the prime time and a few hundred megabytes of RSS, all to re-hold what the log already holds.

The snapshot is now a log-backed store: an index of hash -> log location, and container bytes read from the log on demand as a round emits them, memoised only for the round and released after it. Persisted order is preserved so the clone pager still cursors correctly, the binary-digest to hex-wire-name conversion stays at its one site, and a read fault after the scan surfaces as a server storage error so a clone is never silently truncated. The old whole-store copy is retained only where it is actually needed: the pull client's holdings and the equivalence test's oracle.

Measured, 67499-artifact repository, ReleaseSafe: a clone off the indexed serve is byte-identical to a clone off the copy-based serve (both materialise all 983 files of trunk identically), while serve peak RSS falls from 157 MB to 32 MB. The one remaining O(store) cost, the startup scan itself, is a separate change. check-in: e17845e40d user: dmestas tags: trunk

zit serve built its whole in-memory snapshot of the repository before it opened the listener socket. The build is O(store), so past a few thousand artifacts it outlasted the client's readiness probe and the client reported the server never came up — the store could be served, just never reached.

Bring-up now binds first: listen, announce, then prime, then accept. A client that connects during the prime queues in the kernel accept backlog instead of being refused, so the readiness window starts at the bind rather than after the scans. The listener open moves ahead of BOTH O(store) reads on the path — the config read and the prime. The connection deadline is armed only on accept, so priming never falls inside it. On a 254k-artifact store a connect now completes in ~2 ms against a ~5 s prime.

A push made the snapshot stale, and the old code rebuilt it inside the pushing round's own reply deadline — an O(store) rebuild charged to the request that caused it. The rebuild moves to a maintain hook the accept loop runs between connections, with no connection open to charge it to; the append-only stat gate makes the stale snapshot self-correcting until then.

The reorder is inherently a large-corpus latency property: reverting it leaves every unit test green, because the failure only shows when prime latency exceeds the readiness window. It is pinned in the milestone tier instead, where smoke already lives: zig build bench -- serve synthesizes a store, spawns the real server, and asserts time-to-first-connect is a small fraction of the prime (fixed order ~0.1; reverted order jumps past 1.0). The de-race and the accept-loop hook are pinned by ordinary unit tests. check-in: de218fa6f6 user: dmestas tags: trunk

The 'cli push' and disconnect-robustness scenarios bound a fixed loopback port, so two e2e binaries running at once — the ordinary case under a parallel build or a busy CI runner — collided on it and one push exited non-zero, failing an unrelated-looking assertion. The failure was invisible because the assertion reported only the child's exit code and discarded its stderr.

Each scenario now draws a fresh port from the kernel (bind :0, read back the assigned port, release) and waits for the child to accept before driving it, the same approach freshLoopbackPort takes for the Fossil smoke tests. The push assertion now surfaces the child's stderr on failure, so the next divergence is diagnosable rather than merely counted.

Verified on Linux under the exact condition that reproduced the flake: two 'zig build e2e' runs in parallel, ten rounds, x86_64 and aarch64 — 20 of 20 green on each, where the fixed-port version failed intermittently. check-in: b4a5886252 user: dmestas tags: trunk

Regenerated from the RFC suite after the erratum repo-state-00-s4-unclassifiable-artifact was applied. New §4.4 states what assembly does with a stored artifact classification rejects: an artifact whose octets hash to the name it is stored under is opaque to repository state and MUST NOT fail the assembly, while one whose octets do not hash to its name is a distinct integrity failure and MUST be rejected. The remaining §4 subsections renumber behind it.

The draft was previously silent on this, and that silence was the trap: an implementation conforming to both drafts could reject an artifact per the required-card rule and then treat that rejection as fatal, which made the canonical fossil repository unassemblable. src/update.zig implements the rule this text now carries. check-in: 42e1faa9fc user: dmestas tags: trunk, version-0.1.2

The 0.1.2 release check-in carried the version bump and the README pointer but not the notes themselves: the add that would have tracked them was refused by a locked checkout database, and the refusal was hidden behind a pipe that kept only the last line of output. The file existed on disk and in no commit. check-in: aabc10268a user: dmestas tags: trunk

Two defects made zit unusable against a repository of real size and real history. Serving a sync round cost work proportional to the served store rather than the request, so past a few thousand artifacts a round outran the connection deadline and the server answered nothing at all. Repository-state assembly rejected an entire repository over one stored artifact it declined to classify, while every read verb skipped the same artifact.

Together they meant a clone of the canonical fossil repository could be fetched but not used. It now clones in 67 rounds and materialises all 983 files of the check-in, each byte-identical to fossil's own checkout of it. check-in: 800caecbf2 user: dmestas tags: trunk

A conforming repository contains artifacts a conforming parser declines, and assembly rejected the whole repository over one of them while every read verb skipped it. Bytes that hash to the name they are stored under are stored correctly and merely unclassifiable, so they are opaque to state and skipped; bytes that do not are an integrity failure and stay fatal.

This closes the tolerance half of the update-diagnostics work. On a clone of the canonical fossil repository, update now materialises all 983 files byte-identical to fossil's own checkout. check-in: 6a0f1f3f66 user: dmestas tags: trunk

A conforming repository contains artifacts a conforming parser declines. The artifact grammar requires the U card on a wiki artifact; fossil elides it when the submitter is "nobody", stores the result and syncs it. A clone of the canonical fossil repository therefore carries such artifacts, and assembly rejected the whole repository over one of them while timeline, ls, info and branch read the same store correctly.

The two conditions are separated by the name. Bytes that hash to the name they are stored under arrived and were stored correctly, so the only event is that this reader's grammar does not admit them; the artifact is opaque to state and is skipped, exactly as a non-participating type already is. Bytes that do not hash to their name are an integrity failure and stay fatal. Verification runs before the skip, which is what keeps the second from being swallowed by the first.

On a clone of the fossil repository, 67499 artifacts: update went from exit 1 having materialised nothing, to materialising all 983 files byte-identical to fossil's own checkout of the same check-in. closed check-in: f4736670a8 user: dmestas tags: fix10-tolerance

Two costs made a clone of a large repository O(rounds x store) of work; past a few thousand artifacts a round outran the connection deadline and the server answered nothing at all.

The reply's advertise walk classified every rid by calling content(), which INFLATES a wire-backed entry, so announcing the store cost its uncompressed size every round. It now reads held(), answering from the retained form alone. The send gate still reads content(): uploading composes from plaintext.

ServeCtx caches the snapshot and rebuilds only when a stat shows the log grew, so a commit made alongside a live server is still advertised next request. That rebuild is O(store) and stays out of every client's budget: serve primes before it listens, and a round that appends rebuilds before returning. Reuse needs isolation, since the responder interns phantoms as it answers, so a new OverlayStore gives each round a write overlay. Inflating memoises, so a snapshot outliving its round accumulates the whole repository's plaintext; it is released per round.

The one conflict, at the tail of src/cmd/sync.zig, was two independent test appends; resolved as the union. check-in: 6763d35da7 user: dmestas tags: trunk

update, open, merge and bisect collapsed ApplyError.StateRecordInvalid and ApplyError.Corrupt onto one line, "cannot read repository: repository is corrupt". Corrupt means stored bytes do not match the name they are stored under. StateRecordInvalid means only that a stored artifact was not usable as a check-in state record, which says nothing about integrity. An intact fossil clone reaches the second routinely, so update accused an undamaged clone of corruption while timeline read the same store fine.

StateRecordInvalid now carries its own reason text, shared across the four verbs so they cannot drift or re-merge. The bisect mapper had a blanket else arm swallowing the whole set into the corruption line; it is now exhaustive.

Diagnostics only: assembly still rejects these artifacts, so update still exits 1 on such a clone. The tolerance fix is separate; the shape that triggers it, an anonymous wiki page with no U card, is pinned. check-in: e3bbb4ff22 user: dmestas tags: trunk

Two costs made a clone of a large repository O(rounds x store) of work; past a few thousand artifacts a round outran the connection deadline and the server answered nothing at all.

The reply's advertise walk classified every rid by calling content(), which INFLATES a wire-backed entry, so announcing the store cost its uncompressed size every round. It now reads held(), answering from the retained form alone. The send gate still reads content(): uploading composes from plaintext.

ServeCtx caches the snapshot and rebuilds only when a stat shows the log grew, so a commit made alongside a live server is still advertised next request. That rebuild is O(store) and stays out of every client's budget: serve primes before it listens, and a round that appends rebuilds before returning. Reuse needs isolation, since the responder interns phantoms as it answers, so a new OverlayStore gives each round a write overlay.

Inflating memoises, so a snapshot outliving its round accumulates the whole repository's plaintext. It is released per round. Four clones of 3001 artifacts, ReleaseFast: 12 MB and 0.09 s, against 26 MB and 0.15 s. closed check-in: e2e9233582 user: dmestas tags: fix9-serve-stall

zit update, open, merge and bisect collapsed ApplyError.StateRecordInvalid and ApplyError.Corrupt onto one line, "cannot read repository: repository is corrupt". They are different claims. Corrupt means stored bytes do not match the name they are stored under. StateRecordInvalid means only that a stored artifact was not usable as a check-in state record, which says nothing about integrity.

An intact fossil clone reaches the second routinely: fossil stores and syncs artifacts it declines to crosslink itself. Assembly treats those as fatal while the read verbs skip them, so update accused an undamaged clone of corruption while timeline read the same store fine.

StateRecordInvalid now carries its own reason text, shared across the four verbs so they cannot drift or re-merge. The bisect mapper had a blanket else arm swallowing the whole set into that corruption line; it is now exhaustive over ApplyError.

Diagnostics only: assembly still rejects these artifacts, so update still exits 1 on such a clone. The tolerance fix is separate; the shape that triggers it, an anonymous wiki page with no U card, is pinned. closed check-in: f75dfdfbe7 user: dmestas tags: fix10-update-diag

zit serve rebuilt its whole in-memory snapshot from the log on every request, and classified held-vs-phantom by calling content on every rid, forcing a full inflate of the entire repository per request. A clone cost O(rounds x store) of server work; past a few thousand artifacts one round outran the connection deadline and the server answered nothing at all.

ArtifactStore grows heldCount, the aggregate of the existing non-inflating held predicate, and serve counts with it rather than re-deriving a worse predicate caller-side. ServeCtx caches the snapshot and rebuilds only when a stat shows the append-only log grew, so a commit made alongside a live server is still advertised on the next request.

Reuse needs isolation: the responder interns phantoms and upgrades entries as it answers, and that would change what later rounds serve. A new OverlayStore answers each request over a per-request write overlay, reads falling through, writes confined to the overlay. The persist pass now takes an ArtifactStore.

On 67,499 artifacts: unfixed, no round completes; fixed, one snapshot build then sub-millisecond rounds. check-in: 2874978461 user: dmestas tags: fix9-serve-stall

Harden native Linux filesystem behavior, deterministic repository iteration, update and undo transitions, Fossil smoke-process containment, and the three-platform CI matrix. check-in: be97df0398 user: dmestas tags: trunk, version-0.1.1