* fix(manifest): maintain the leaf table incrementally (RS-7.3-3)

The leaf table was recomputed from scratch at the end of every crosslink batch, so a check-in lost its leaf verdict whenever a later batch linked an untagged child to it. repo-state-01 Section 7.3, as corrected, and fossil keep that verdict: a check-in and its plink parents are rechecked only when a branch tag is written to it or propagation updates or deletes its branch row (src/tag.c:195, :100-102), and the checks run when the batch ends (src/leaf.c leaf_do_pending_checks).

The rule now lives in a new internal/leaf package: Check (leaf_check), Rebuild (the explicit full rebuild) and Pending (the deferred schedule). internal/tag schedules a recheck at exactly the branch-row writes; the crosslink sweep, receive linker and repair gate apply a batch's schedule at the gate, each phantom-fill cascade transaction and each commit or tag transaction apply their own before commit, and verify.Rebuild keeps the whole-table recompute. Reparenting schedules nothing, as in fossil, so the reparent case is split into a one-batch case and a per-artifact case.

Tracked in #426.

* fix(manifest): recheck leaves a cancel reaches; drop rechecks of a failed receive tx check-in: 628f932301 user: [email protected] tags: main, trunk

* test(conformance): re-pin the -01 drafts with the 2026-10-13 errata

Re-pin the repo-state and sync-protocol drafts to zit check-in 5a1b741b25d8, which applies five more -01 errata: repo-state §5.3, §7.3 and §8.3, sync-protocol §9.4/§9.5 and §10.1. zit now keeps each -01 text under the -00 file name; the copies keep their -01 names and conformance/rfcs/README.md records the zit path and check-in. The new errata are pinned, and the requirement tables are re-read against every changed hunk.

The §7.3 and §8.3 cases that were deviations now pass as written, and new cases cover a date tag's arrival order (§5.3), untagged leaves and a merge parent's recheck (§7.3).

The sync server answers each guard-passing ci-lock with pragma avoid-delta-manifests while forbid-delta-manifests is on, defaulting to on as the draft and fossil 2.29 do (fossil 2.28 defaults it off). The client leaves its transfer unchanged on that pragma. A file or cfile whose content does not match its hash now fails the client's round, in a pull and a clone: the reply rolls back, prefix included, and no further round starts.

The leaf table is still recomputed per crosslink batch, so a parent whose untagged child arrives in a later batch loses the verdict §7.3 keeps; that case is marked as a gap, tracked in #426.

* fix(sync): continue after a uvfile content error; renumber the avoid-delta-manifests row

A uvfile whose content does not match its HASH is a handler-reported content error: warn, commit the prefix and follow ordinary continuation (sync-protocol-01 10.1). The avoid-delta-manifests client row is SP-9.5-7 now that share-links took SP-9.5-5 and SP-9.5-6. Older unreleased changelog entries drop the pending-erratum wording. check-in: 600861484f user: [email protected] tags: main, trunk

* fix(sync): repair leaves and tags for check-ins the cascade links (#414)

A pulled check-in arrives as an announced phantom filled the next round, so the phantom-fill cascade links it. The session's closing sweep then found nothing left to link and skipped the leaf and tag-propagation repair, which gated on the sweep's own count: the check-in had an event row and no leaf row, and its parent stayed a leaf. The server handler had the same gap for a pushed artifact that fills a phantom.

AfterDephantomize and AfterDephantomizeBatch now return how many artifacts they linked. The sync client and the handler add that up over the session and close it with the new CrosslinkAfterCascades, which runs the repair once when the sweep or the cascades linked anything (repo-state-01 Section 7.3, RS-7.3-4), not once per file. The receive linker's cancellation paths gate on the same flag.

The repo-state harness gains a pull load: the first artifacts of a script are held and crosslinked locally, the rest pulled in process from a go-libfossil server holding them all. New cases repostate.7.3.pulled-child and repostate.7.6.pulled-retag, checked against fossil per artifact, each run as a .pull case too.

Tracked in #414.

* fix(sync): repair leaves of cascade links when a session fails (#414) check-in: 0035af8ac5 user: [email protected] tags: main, trunk

* fix(manifest): link check-ins with absent files; refuse absent baselines

The crosslink sweep held a check-in back until every file it names was stored. A root check-in, or one on a present parent, whose files never arrived got no event, plink, mlink, leaf or tag rows. fossil links it against phantom files (add_one_mlink, uuid_to_rid(..., 1)), and repo-state-01 Section 4.2 makes an unresolved reference a phantom, not an error. The sweep -- Crosslink, the close of every sync and clone, and verify.Rebuild -- now waits only on a delta manifest's baseline. The receive-time cascade and receive linker still wait for files while blobs are still arriving; the closing sweep bounds that wait.

verify.Rebuild linked a delta manifest whose baseline was absent, with no baseline id. fossil refuses it (fetch_baseline) and files it in orphan under a phantom for the baseline; artifact-format-01 Section 8.3 calls this a soft failure retried when the baseline arrives. Every path now does the same, so a sync requests the baseline and the phantom-fill cascade links the manifest when it lands. Rebuild clears and rederives orphan.

The gap case repostate.4.2.root-absent-file passes and is unmarked; new cases cover a present parent with an absent file and an absent or late baseline, each also through verify.Rebuild where fossil rebuild agrees.

Tracked in #406.

* fix(manifest): refuse a non-manifest baseline and drop the orphan row on link

Tracked in #406. check-in: 67fc6ced4b user: [email protected] tags: main, trunk

* fix(manifest): link a check-in atop an absent parent despite absent files

The crosslink deferral held back a check-in until every F-card blob was stored. When the primary parent's manifest is itself absent, no mlink row depends on those files (fossil's add_mlink returns before reading one, and the parent's crosslink derives the transition once it arrives), so the wait could never complete for files the repository will not receive. The libfossil repository holds one such check-in, a fossil-scm commit imported without its history; the sweep lost its event and plink rows that fossil rebuild writes (RS-4.2-2).

missingCheckinRefs now skips the F-cards when the primary parent is unavailable; the baseline is still required. Adds conformance case repostate.4.2.phantom-parent-absent-file and a unit test that also checks the child's mlink is derived once the parent arrives.

Tracked in #398.

* fix(manifest): record the remaining crosslink deferral gaps

Adds gap case repostate.4.2.root-absent-file, a receive-linker ordering test for the child-before-parent path, and notes the narrowed exemption. Tracked in #398 and #406. check-in: 15762adeb6 user: [email protected] tags: main, trunk

* fix(manifest,hash): verify the R card at commit; reject an unrecognized hash-policy

A check-in reads its manifest back and requires the R card to equal the digest recomputed over the resolved tree, byte for byte (artifact-format-01 6.3.4, AF-6.3-5). A hash-policy setting outside the ordinals 0 through 4 is now an error instead of a fall back to the default (7.1, AF-7.1-2).

Tracked in #387.

* fix(manifest,repo): drive the commit R check from conformance; refuse a bad hash-policy on write check-in: c42a2b90c6 user: [email protected] tags: main, trunk

* test(conformance): cover artifact K, delta-lookup and delta-size requirements

Single-name lookup in a delta manifest now follows artifact-format-01 8.4 (manifest.LookupFile), Marshal writes the K ticket ID lowercase (4.7.9), and new cases cover 3.1 size limits, 3.4 checksum mismatches and date-only timestamps. AF-6.3-5 and AF-7.1-2 stay uncovered with a reason in their rows.

* test(conformance): cover artifact K, delta-lookup and delta-size requirements (review fixes) check-in: 22b3a0f06d user: [email protected] tags: main, trunk

* fix(tag): derive the check-in event time from direct date tags only (#372)

Section 4.2, step 6 of repo-state-01 makes a date tag applied directly to a check-in set its event time, whatever order the tag and the check-in arrive in. Crosslinking read the date tagxref row whatever its source, so re-crosslinking after a propagating date tag retimed the descendants that inherited it, and a cancel, an unparsable value or 0 left no time or a time of 0, as fossil 2.28 does.

tag.RefreshEventTime now holds the rule, and both tag application and crosslinking call it: a direct, non-cancel date row whose value parses to 0 < r < 5373484.5 sets mtime; anything else restores the D-card time kept in omtime. The date-tag cancel, unparsable and zero cases are no longer deviations, and new before-checkin cases cover the other arrival order.

Tracked in #372.

* fix(tag): word the date-tag parse rule as step 6 does (#372)

* docs(changelog): tidy the entry spacing (#372) check-in: 30b9d65021 user: [email protected] tags: main, trunk

* fix(manifest): resolve a delta manifest's baseline one level (#371)

A B card may name another delta manifest (artifact-format-01 4.7.3). Drop the #275 refusal in loadBaseline, which ListFiles and crosslink share, and use the named manifest's own F cards.

Tracked in #371.

* fix(manifest): drop baseline delete markers and never write delta-of-delta (#371)

* docs(changelog): note delete markers and the writer rule for delta baselines (#371) check-in: 62396a79e5 user: [email protected] tags: main, trunk

* test: isolate fossil's global config in tests that run the fossil binary (#284)

Tracked in #284.

* test(testutil): document the file-level scope of the fossil home guard (#284)

* test(testutil): allow StartServer in the fossil home guard (#284) check-in: 8d3f18265e user: [email protected] tags: main, trunk

* fix(tag): apply bgcolor, user and comment tags to event as fossil does (#343)

tag.ApplyTagWithTx now sets event.bgcolor, euser and ecomment from the bgcolor, user and comment tags for every tag type, NULL on a cancel, as tag_insert does. Crosslinking a check-in reads the three values back from tagxref, as manifest_crosslink does, so a tag that arrived while the check-in was a phantom takes effect.

Tracked in #343.

* fix(tag): store empty event override values as NULL (#343) check-in: 66cd641642 user: [email protected] tags: main, trunk

fossil's client_sync continues a round when stored content or a new phantom leaves the phantom table non-empty, or when files are queued for the next request (section 3.7). The client instead continued on any received file and on any phantom until it had aged three rounds, kept a gimme it could not answer queued forever, and kept an igot's phantom only in the session's map. After a clone's first round it aborted on a server error, where the draft ends the reply-card loop and goes on.

converged now applies 3.7 (a) and (b). A gimme is queued only for content the client holds and may send. An igot's phantom is written to the phantom table, batched once per reply as content_new would write it, and the after-dephantomize cascades of a reply run as one batch (manifest.AfterDephantomizeBatch), so a pull no longer begins a crosslink transaction per artifact. A clone records a later round's server error, stops reading that reply, continues by 3.7, and fails at the end naming the error. Four unit tests that encoded the old continuation now expect fossil's. check-in: 62683d1d0f user: [email protected] tags: main, trunk

branch.Create refused a name an open leaf carries (fossil's branch_is_open), but checked before manifest.Checkin opened its transaction, and Checkin applied the branch tags, parent propagation and leaf state in later transactions. Concurrent creates of one name could all pass the check and all commit.

Checkin now derives its tags and leaf state inside its own transaction and runs an optional Precheck first in it. Create checks the name there, so check and write are one transaction: with BEGIN IMMEDIATE the second writer sees the first's open leaf and is refused. Create's redundant second tag pass after the check-in goes away. check-in: 9c071c7c22 user: [email protected] tags: main, trunk

fossil's rebuild_db empties and re-derives the repository's tables in a single transaction. Since #285 Rebuild committed the emptied derived tables before the crosslink sweep, because the sweep found candidates outside any transaction and opened its own; a sweep that failed partway left event, plink, mlink, tagxref and leaf empty.

manifest.CrosslinkRebuild now runs inside the caller's transaction: candidate discovery, linking and the leaf/tag repair gate all on tx. Rebuild empties, re-derives and refills the bookkeeping tables in one WithTx, so a failure rolls back to the state it found. The incremental sweep keeps its batched commits; only the repair gate is factored out so both paths share it. check-in: 2832099f56 user: [email protected] tags: main, trunk

fossil's check-in time override (`fossil amend --date`) is a one-shot date tag. manifest_crosslink takes a check-in's event.mtime from that tag's value when tagxref already holds one, and tag_insert moves event.mtime (keeping the original in omtime) whenever a date tag wins its tagxref row. Crosslink always used the D-card time, so every date-amended check-in sat at its original time.

Crosslinking a check-in now reads its date tag back, and tag.ApplyTagWithTx applies the override. tag.ApplyTag and tag.AddTag now derive through ApplyTagWithTx rather than repeating it, so a tag added directly moves event.mtime and event.bgcolor as crosslink would.

New fossil-referenced repository-state cases cover the tag after the check-in, before it (on a phantom), and newest-wins. check-in: 81b0b585e5 user: [email protected] tags: main, trunk

deck.Parse determined an artifact's type but then checked nothing against it, so a card a type disallows, a required card left out, or a tag that breaks a per-type semantic rule all parsed.

Add deck/legality.go with fossil's manifestCardTypes table (allowed and required cards per type) and the §5.4 semantic checks: a control artifact's tags may not target itself, a technote's tags are all self-referential singletons, and a forum post carries at most one of an H or I card and at most one parent. Parse now determines the type (inferType gains the B, J and Q type-setters it was missing and falls back to a control artifact, not an empty check-in, when nothing sets a type and there is no comment or parent), then applies legality and the semantic checks. A missing Z card is now caught by the universal required-card rule.

Several internal/manifest and internal/deck tests built incomplete artifacts fossil also rejects -- a cluster with a D card, a forum post with both an H and an I card, control-shaped D/U/Z artifacts with no T card, and single-card fixtures missing their type's required cards. Frame each as a valid artifact of its type; the behavior each test pins is unchanged.

This also makes two repostate conformance cases conform: the technote with a non-self tag is now rejected at parse (not by a later isArtifact gate), and the opaque-control rebuild case. Their marks are removed in conformance/repostate_gaps_test.go. check-in: 7b8f655da6 user: [email protected] tags: main, trunk

verify.Rebuild kept its own partial derivation of repository state: it wrote event/plink/mlink/tagxref rows only for check-ins and control artifacts, applied tags from artifacts the crosslink would refuse, skipped absent tag targets and parents instead of reserving phantoms, wrote no plink baseid, and computed leaves as "no child at all" regardless of branch. Against fossil rebuild it missed hundreds of event, plink, tagxref and leaf rows on real repositories.

Make Rebuild empty the crosslink-derived tables and run the same sweep a clone runs, so derived state has a single owner. The three partial derivation files (rebuild_tags.go, rebuild_manifest.go, rebuild_leaves.go) are deleted; Rebuild keeps only what the sweep does not do -- blob hash verification and the unclustered/unsent bookkeeping.

A rebuild is terminal: nothing more will arrive, so a check-in whose file or baseline content is absent must be linked with phantoms rather than deferred for a blob that never comes, as fossil's add_one_mlink does via uuid_to_rid(...,1). Add manifest.CrosslinkRebuild, a sweep that does not defer; on a complete repository it is identical to Crosslink. Rebuild's leaf rule now follows fossil's branch-aware sweep, so an untagged parentless check-in is no longer a leaf -- the same Section 7.3 deviation the sweep and per-artifact paths already carry. check-in: 33959a6e3e user: [email protected] tags: main, trunk

The parser stored hash-bearing tokens without checking them, so an F, B, E, G, I, K, M, P, Q, A-source or T-target token of the wrong length or alphabet passed, and the A target and N mimetype were kept raw rather than decoded.

Validate each position against the §4.7.1 map: 40 or 64 case-insensitive hex for an artifact hash, 40 for the K-card ticket id, 32 for the R checksum. A baseline manifest now requires a hash on every F card; only a delta manifest (one with a B card) may carry a hashless deletion card. Decode the A target and the N mimetype per §4.3.1 and re-encode them on marshal, keep the other hash tokens raw, and keep only the final path component of the A filename. Reject a Q sign other than + or -, and a tag name that is entirely hex in either case.

Four internal/manifest crosslink tests built manifests fossil itself rejects -- two R cards 40 hex long (R is 32), an attachment source "abc123", and a baseline manifest with a hashless deletion card. Make those fixtures valid artifacts (32-hex R, a real source hash, the merge expressed as a delta off its primary parent so the deletion card is legal); the mlink and crosslink outcomes they assert are unchanged. check-in: 1a15366e29 user: [email protected] tags: main, trunk

* fix(manifest): apply the parent (reparent) tag (#286)

A singleton parent tag relocates a check-in's ancestry. fossil's manifest_crosslink_end rewrites the tagged check-in's plink and mlink to the hashes the tag names, before it propagates tags, so the leaf recompute sees the new edges and an old parent with no remaining child becomes a leaf. ReparentCheckins does this over a deck whose P-card is the override, reusing the normal plink/mlink builders. It runs from the crosslink repair gate and from verify.Rebuild, which had no reparent step.

* test(manifest): reparent plink/mlink/leaf, idempotence, phantom target (#286)

Also fold the plink baseid resolution into a shared checkinBaseid helper used by both the normal linkage path and the reparent pass, and note that the reparent repair is O(reparented) per round like fossil's manifest_crosslink_end. check-in: 088f1d8a90 user: [email protected] tags: main, trunk

fossil rejects a CFTYPE_EVENT artifact at parse time unless every T-card is a '+' tag on the technote itself; a technote may not tag another artifact, propagate, or cancel. go-libfossil's parser still accepts such a technote, so gate it in isArtifact: a technote whose T-cards are not all self-referential singletons is not an artifact and applies none of its tags. check-in: cf9089ec35 user: [email protected] tags: main, trunk

* fix(manifest): make root check-in mlink derivation idempotent (#294)

The root-file branch of insertCheckinMlinks inserts each file row with no already-derived guard, unlike addMlink's pmid>0 early-out, so crosslinking a root a second time -- which the relink path does after clearing event and tagxref -- inserted a second copy of every row. Skip the block when a root row (pmid=0) is already present, matching the idempotence of fossil's manifest_crosslink.

* test(manifest): root check-in mlink derivation is idempotent on relink (#294) check-in: d5fdd30d12 user: [email protected] tags: main, trunk

fossil new and repo-state §6.1 write *branch and *sym-trunk, both propagating, so the trunk alias descends to every check-in on the branch. The initial check-in built sym-trunk as a singleton (+), which stopped the alias at the root. check-in: 470f85d9fa user: [email protected] tags: main, trunk

ListFiles took any parseable artifact named by a B card as the baseline, so a delta manifest or a control artifact produced a partial or empty file set. Sections 4.7.3 and 8.3 require the target to be a baseline manifest: a checkin with no B card of its own. ListFiles now returns an error otherwise. check-in: 9fbe647b7f user: [email protected] tags: main, trunk

Checkin wrote any F-card filename it was given. The artifact format's writer profile (4.6.1) forbids a backslash and anything but valid UTF-8, surrogates and overlong forms included; fossil rejects a manifest that names one. storeFileBlobs now refuses such a name or old name before storing anything. check-in: 357d10dac6 user: [email protected] tags: main, trunk

The delta format holds integers below 2^32 only, so a producer must not write a target length or copy offset at or past that. Create now returns ErrTooLarge for such inputs; Deltify then stores the content whole, as fossil does when a delta is not worthwhile, and the other callers propagate the error. Apply's header check becomes >= so it refuses exactly what Create will not write. The bound is tested on lengths, as a direct test needs a 4 GiB input. check-in: 38eddf43f1 user: [email protected] tags: main, trunk

* fix(manifest): store check-in, link and tag rows as fossil does (#258)

A rebuilt or locally written repository differed from fossil's in three derived values: event and plink times in their last digits, event.omtime left NULL, and a valueless tag stored as '' instead of NULL.

Fossil stores a tag time as SQLite's julianday() bound as a double, but writes event, plink, attachment and forumpost times into SQL text with its own printf("%.17g"), which emits 16 digits in long double arithmetic. TimeToJulian is now julianday() exactly (whole milliseconds over 86400000) and TimeToTextJulian ports fossil's printf conversion, so both match fossil bit for bit. The conversion follows builds where long double is a double (macOS, Windows); fossil itself differs across platforms in the last digit.

Check-in events record omtime, tag writes bind NULL for an empty value, the readers of event and plink times under the ncruces driver rebuild the text form, and the CLI's tag listing reads NULL values.

* test(db): sweep both stored julian forms; scan the text form from driver times (#258)

* test(verify): compare text times bit for bit only where fossil computes them in double (#258) check-in: 2c2e5e74b6 user: [email protected] tags: trunk

SeedConfig never wrote hash-policy, so naming fell back to deriving from the repo's own artifacts — and an empty repo has none, settling on SHA1 permanently. Seed sha3, which is what fossil init writes.

Clone drops the row the same way it drops project-code: a clone adopts its source's policy, and an absent row derives naming from the artifacts that arrive.

Moves the policy enum to db so the seed and hash.NamingFor read one definition, and updates fixtures that hardcoded SHA1 names to ask the repo how it names artifacts. check-in: 1c02c944ec user: [email protected] tags: trunk

Store hardcoded SHA1, so committing into a sha3 repo mixed hash families and made every tracked file read as CHANGED. Read hash-policy once per store and name new artifacts with the algorithm it selects.

Content already stored under the other algorithm's name keeps that name unless the policy forbids it (sha3-only, shun-sha1), so unchanged files don't churn their F-cards.

Also routes checkout.Manage and uv.Write through the same lookup, replacing their width-sniffing guesses — uv's read project-code, which is 40 hex chars in every repo, so it could only ever pick SHA1. check-in: 847c195286 user: [email protected] tags: trunk

Keep validated receive bytes in a bounded session linker so clone does not expand and hash every artifact again during finalization. Preserve bounded retry, cancellation, replay, and residual-repair behavior.

Closes #214 check-in: 7b6a67fb81 user: [email protected] tags: trunk

Route direct check-ins and internal rebuilds through canonical mlink derivation. Update sparse-history consumers and preserve legacy conflict-fork rows with an explicit RID discriminator migration.\n\nCloses #213 check-in: 04f41a4c9c user: [email protected] tags: trunk

Cloning the Fossil SCM corpus (67.6k blobs) took 293-302 s at 1251-1287 MB against canonical fossil's 20.8 s at 205 MB. A CPU and allocation profile of one clone found the cost was not delta expansion or hash verification.

27.2% of all CPU was a single un-memoized statement. content.AvailabilityCache memoizes the delta-chain walk but not the "SELECT rid FROM blob WHERE uuid=?" that starts it, so manifest.missingCheckinRefs re-seeked the blob.uuid index once per F-card of every check-in -- the same file hash resolved again for every revision that did not change it. That index is far larger than SQLite's page cache, so each seek was three page reads: pread syscalls were 34.7% of total samples. The walk it guards was under 1% of its own cost.

Separately, materializing 8.27 GB of content allocated 360 GB. delta.ApplyInto starts at a 64 KiB cap deliberately -- the target length is a wire claim and a few bytes can declare 4 GiB -- and then climbs to the real size through append's ~1.25x large-slice growth, ~3.4x the target in discarded intermediates, leaving slack that content.Cache.clip copied away again (203.9 GB and 59.8 GB respectively).

Three changes:

- Memoize uuid -> rid in AvailabilityCache. Hits only. Within one crosslink run a hit cannot go stale, because a sweep and a cascade only ever add blob rows; a miss can, because ridOrPhantom and blob.StorePhantom create rows mid-run, and remembering an absence would leave an artifact deferred after the content it waits on arrived. The scope that makes this sound is now stated on the type, and TestAvailabilityCacheMustNotSpanAPurge pins its one boundary: shun.Purge deletes blob rows, so a cache must not span one.

- Size delta.ApplyInto's output exactly. scanOutputSize walks the command stream first without allocating, checking every copy against bytes the source holds and every insert against bytes the delta carries, so the size it returns is a measurement rather than the header's claim. Any stream it cannot walk to a clean terminator falls back to the existing growth path, so malformed and hostile deltas allocate and fail exactly as before. The buffer now arrives tightly sized, which also makes clip's copy a no-op.

- Cut crosslinkCacheBytes from 256 to 128 MiB. Each byte of that budget costs roughly five bytes of peak RSS -- GOGC doubles it into the heap goal, and the scavenger is paced off GC cycles, of which a faster sweep runs fewer. At 256 MiB the first two changes gave 205.7 s but 1473 MB, faster at the cost of memory. The measured curve (32/64/128/256 MiB: 369.6 s/318 MB, 292.3 s/447 MB, 249.4 s/812 MB, 205.7 s/1473 MB) puts 128 MiB where both axes improve.

Result on the same corpus: 245.4 s at 780 MB, from 293-302 s at 1251-1287 MB -- 17% less wall time and 39% less peak RSS, with instructions retired down from 3.37T to 2.48T. The gap to canonical narrows from 14.1x/6.3x to 11.8x/3.8x.

Derived tables are unchanged against an unmodified build cloning the same server: blob 67530, event 27099 (identical per type), mlink 104919, plink 23741, tagxref 51766, leaf 1426, filename 1496. The corpus parity gate passes.

What remains is structural rather than a missing optimization: a clone expands every blob twice, once in the receive-time phantom-fill cascade and once in the final whole-repository sweep, where canonical crosslinks each artifact from content it already holds in hand at receive time. Cache.Expand splits 45.4 s / 35.3 s across those two passes. Collapsing them is a change to the receive path and belongs in its own issue. check-in: 8b16c91f5c user: [email protected] tags: trunk

A full clone of the Fossil self-hosting repository disagreed with what canonical fossil derives from the SAME artifacts: mlink -3,351 rows, tagxref -632, plink -5, leaf -1. `fossil rebuild` on our own clone put every one of them back, so the artifact set was complete and the fault was entirely in crosslink-time derivation.

Six independent causes, each verified against `fossil rebuild` on a 67.6k-blob corpus clone:

mlink, -3,351. An mlink row records a parent-to-child TRANSITION, and a merge check-in has more than one parent. Canonical runs the same file-level diff once per parent -- primary, then each merge parent, then each cherrypick source -- and flags the extra rows isaux=1; we only ever diffed against the primary parent. add_mlink is now ported whole (internal/manifest/addmlink.go), which also brings the two passes the old code approximated: the delta-vs-delta revert/resurrect pass, and the baseline-child deletion pass with its mperm=0 delete rows. "Added by merge" (pid=-1) follows canonical's rule too -- a count over the rows a file produced, not a name lookup in the merge parents. A parent whose manifest cannot be read now yields NO rows rather than a whole tree of bogus "added" rows, and the rows are derived later from the parent's side once it arrives (linkPlinkChildren), so the result no longer depends on the order the sweep visits artifacts in.

tagxref, -612 of -632. A check-in's T-card does not have to name the check-in carrying it: `fossil merge --integrate` records the branch it closed as `T +closed <hash>` on the merge check-in itself. Canonical applies a check-in's, a control artifact's and a technote's T-cards through one code path; we applied only the self-referential ones on check-ins and none at all on technotes.

tagxref, the rest. A wiki-

or event-<id> tag's value is the W-card body measured after LEADING whitespace is skipped, not the raw card length.

Artifact identification. The sweep parses every blob in the repository, so what counts as an artifact decides which rows exist at all. Three gaps: a bare card letter with no arguments is legal (fossil's own first check-in spells its parentless-ness as a bare "P" card) and rejecting it dropped that check-in entirely; an N-card after a P-card is legal in a technote for historical compatibility, and rejecting it dropped real technote revisions with their event-<id> and bgcolor tags; and the check-in-versus-control decision turns on a comment, files or parents, which made `merge --integrate` check-ins read as control artifacts. Fossil's required-card table now gates linking (isArtifact), on both the sweep and the dephantomize cascade, so a file whose first line parses as a Z-card no longer links as a dateless check-in and a wiki revision missing its U-card no longer links at all.

plink, -5. The ancestry a check-in declares is a property of the check-in, not of what happens to be in the repository when it is crosslinked. A parent hash with no blob yet now reserves a phantom the way canonical's uuid_to_rid does, so the edge exists whether the parent arrives in a later round or never; skipping it lost the edge for good, because nothing revisits an already-linked artifact.

leaf, -1. Nothing calls leaf_rebuild() outside `fossil leaves --recompute`; the table is maintained by leaf_check() over the rids leaf_eventually_check() schedules -- each rid a branch tag lands on, plus that rid's parents. Drawing candidates from event instead both missed phantom parents (which fossil does list) and added check-ins that carry no branch tag and no ancestry (which it does not).

After: a fresh corpus clone and `fossil rebuild` of that clone agree row for row on mlink, plink, tagxref, leaf AND event -- 104,919 / 23,741 / 51,766 / 1,426 / 27,099 either way, and `fossil test-integrity` reports 0 errors.

Not addressed here: the local commit path (manifest.insertMlinks) still writes one mlink row per file rather than per change, and does not record merge-parent transitions. That is a separate divergence from the clone parity this issue is about.

Co-authored-by: Dan Mestas <[email protected]>

A pull into an already-up-to-date clone of the Fossil SCM repository took 2m44s and moved nothing. A CPU profile of that no-op pull puts 134.4s of 149.1s of samples (90%) inside manifest.CrosslinkContext, the whole-repository sweep Sync runs after the round loop. Nothing in the protocol path registers.

Two independent costs, both O(repository size) and neither bounded by what arrived:

collectCrosslinkCandidates 99.5s (67%) linkBatch -> Cache.Expand 34.9s (23%) of which hash.SHA3 26.0s (17%)

The query was quadratic. Its four exclusions were correlated NOT EXISTS subqueries, and tagxref is indexed on (rid, tagid) and (tagid, mtime) -- neither covers srcid -- so SQLite rescanned all 51,134 tagxref rows once per blob, 67,615 times. EXPLAIN QUERY PLAN showed the CORRELATED SCALAR SUBQUERY -> SCAN tx. Rewritten as NOT IN, each set materializes once and is probed: identical 39,727 rows, 47.7s -> 0.08s measured under canonical fossil's SQLite.

The candidate set never shrank. A blob qualifies when it has none of the derived rows a crosslink writes, and ordinary file content never gains those rows however often it is examined -- so 39,727 of the repository's 67,615 blobs were re-expanded (delta chain walked, SHA3 re-verified) and re-parsed on every sync. A probe run confirms the sweep linked exactly 0 artifacts on each of two consecutive runs over the converged repository, at 3m20s and 3m15s.

Canonical fossil has no equivalent cost because it never sweeps: xfer.c attempts one crosslink per blob as it arrives, artifact or not. This gives the sweep the same once-per-blob property via crosslink_nonartifact, an on-demand table recording blobs that expanded but hold no parseable artifact. Both facts are properties of immutable blob content, so the verdict cannot go stale; a blob that failed to expand, or that parsed but was deferred, is deliberately not recorded and is reconsidered next sweep.

Measured on a clone of the Fossil SCM repository (67,615 blobs), no-op pull:

before 2m43.9s after 0.40s

Clone is unchanged at 5m30s (it pays the one-time examination either way), and the two clones' derived tables are identical -- blob, event, mlink, plink, tagxref, filename and leaf totals all match row for row.

Fixes #202 check-in: 3748b95de7 user: [email protected] tags: trunk

Branch tag propagation left 1,155 of 20,859 check-ins on a clone of the fossil repository carrying a different branch value than canonical, and tagxref at 68,883 rows against canonical's 51,764. Two independent defects in the port, both confirmed against fossil's own source and a canonical rebuild of the same blobs.

tag_insert is mtime-guarded and our tag application was not. Canonical opens it with

SELECT 1 FROM tagxref WHERE tagid=%d AND rid=%d AND mtime>=:mtime

and returns without touching tagxref -- and without calling tag_propagate -- when it matches (src/tag.c:173-186). AddTag, ApplyTag and ApplyTagWithTx all wrote unconditionally, so a check-in's own inline T-cards clobbered a later control artifact that had retagged the same check-in, then re-propagated the stale value to every descendant. On the real corpus one branch rename did this 19,115 times: check-in ab0a0d76 declares branch dual-license, three control artifacts later rename it to clear-title, and we reinstated dual-license and flooded the repository with it. The guard makes the outcome independent of the order the sweep visits the two artifacts in.

repairTagPropagation seeded from the wrong node. tag_propagate's per-child test is strict --

coalesce(srcid=0 AND tagxref.mtime<:mtime, 1) AS doit

-- so a descendant that already carries the tag holds it at exactly the origin's mtime, doit is 0, and the walk neither retags that child nor queues it. Replaying from tag origins therefore stopped dead at the first already-tagged descendant, stranding any check-in linked by a later sweep. Canonical seeds from the primary parent instead, right after inserting a new check-in's plink edges (src/manifest.c:2300-2302 and 2467-2469), which only ever tests that parent's own children. Seed from every primary parent, which is the sweep-order-independent equivalent. Worth 693 missing propagated branch rows.

Measured on a clone of the fossil repository against a canonical rebuild of the same blobs:

divergent branch values 1,155 -> 1 (of 20,858 compared) tagxref rows 68,883 -> 51,132 (canonical 51,764) leaf rows 1,463 -> 1,425 (canonical 1,426)

The residual 632-row tagxref gap is 613 missing `closed` singletons plus small change; it measures identically before and after this change and is a separate defect. The one remaining divergent branch value is a NULL-vs-empty-string difference in cancel-tag values, also pre-existing.

repairTagPropagation still runs before repairLeafTable, as #197 requires. check-in: 7043d6e3a9 user: [email protected] tags: trunk

* fix: derive leaf from fossil's branch rule and bound the deferral rollup

Leaf derivation dropped 66% of rows on the real Fossil repository (487 of 1,426) because both leaf paths treated any parent link as retiring a leaf. Canonical fossil retires a check-in only when a child carries the *same* branch name, with an absent branch tag reading as "trunk" (src/leaf.c: is_a_leaf / leaf_check / leaf_rebuild all share that predicate and none consult plink.isprim). Verified against fossil 2.28 itself: after a branch commit off the trunk tip, both the incremental table and `fossil rebuild` keep the trunk tip as a leaf even though its only child is its primary child, and a branch tip merged into trunk stays a leaf as well.

Two code paths were wrong, in the same way:

- repairLeafTable excluded every `pid` in plink, so merged branch tips disappeared -- the class that accounts for 939 of the corpus's leaves. - Checkin deleted every parent from leaf, so committing onto a new branch silently retired the branch point.

Both now share leafCheck/repairLeafTable in internal/manifest/leaf.go. The recompute also has to run after tag propagation rather than before it: a check-in that declares no branch of its own only has a branch value once propagation has written it, and comparing against a missing value made every inherited-branch tip look like a leaf.

Applying the new predicate to the real repository's own event/plink/tagxref yields 1,428 against fossil's 1,426, and every one of the six differing rows is fossil's incremental bookkeeping rather than its predicate: 2 are phantom rids fossil's leaf_check inserts as plink parents of check-ins it linked, and 4 are parentless check-ins with no branch tag, which nothing ever enqueues for a leaf check. Running fossil's own leaf_rebuild() on that repository gives 1,494, so it is leaf_check, not leaf_rebuild, that the shipped table reflects.

Also caps the deferral rollup's identifier lists (issue #194). A clone deferring 180 check-ins on 417 missing blobs rendered a single ~28 KB log line; the counts stay exact at INFO and the lists print eight entries plus an "...and N more" so nothing is dropped without saying so.

Fixes #189 Fixes #194

* chore: retrigger code scanning after branch restore check-in: df495ba69e user: [email protected] tags: trunk

* fix: write the ticket event row with its tag (#184), quiet the cascade (#186)

crosslinkTicket applied the tkt-<uuid> tag and returned the event row as a pending item for a second pass that was a no-op stub, so the row was never written at all. collectCrosslinkCandidates excludes any blob that already has a tagxref row naming its own rid as srcid, so the tag the incomplete operation wrote also hid the artifact from every later sweep. Cloning fossil's own repository produced 96 event rows of type 't' against canonical fossil's 3,808 -- 100% of the D,J,K,U,Z ticket-change shape lost, with nothing logged and test-integrity still clean, because only derived tables were wrong.

Implement the ticket rebuild instead of reordering the writes: there was no event write to reorder. ticketRebuildEntry replays every artifact carrying the ticket's tag in mtime order and derives its ticket, ticketchng and event rows, mirroring fossil's ticket_rebuild_entry, ticket_insert and manifest_ticket_event. It runs inline in the transaction that writes the tag, so the marker meaning "already linked" and the rows that make that true commit together or not at all. Replaying the whole tag set per artifact also makes the result independent of arrival order.

The event wording, the ticket/ticketchng column mapping including '+field' append semantics, the %h escaping and the two hyperlink hash lengths were all derived from the fossil binary rather than assumed, and are pinned by TestFossilBinaryTicketParity: a corpus built with `fossil ticket add`/`change` is crosslinked by us, rebuilt by fossil, and the two derivations compared row for row. TestCrosslink_TicketArtifactsProduceEventRows is the count assertion whose absence let this ship.

Audited the other three exclusion predicates for the same split-write shape: forumpost and attachment are each written by the same function, in the same transaction, as the event row that accompanies them, and propagate errors, so neither can diverge. No change made to them.

Secondary (#186): AfterDephantomize warned once per blob that failed to parse as a manifest, which is the ordinary outcome for file content and what the whole-repository sweep has always skipped silently -- ~39k lines on a fossil clone, burying the failures that matter. It now skips them the way linkBatch does. A blob that parses but fails to link is still a real fault and still warns.

* docs: record the ticket rebuild's measured cost and why it stays inline

The inline rebuild is quadratic in one ticket's artifact count. The cheaper deferred shape stays atomic but needs two call sites to remember to invoke it, and a forgotten call reintroduces #184 silently. Write the measurements and that reasoning down at the function, so a later profile of a repository with one hot ticket does not turn into an optimization back into the bug. check-in: 39f646e0ae user: [email protected] tags: trunk

The whole-repository crosslink sweep holds back a checkin whose referenced blobs (F-cards or B-card baseline) have not arrived locally, via deferCheckin, so a later sweep can rediscover and complete it once the blob shows up. The mid-round phantom-fill cascade (afterDephantomize) had no equivalent guard and linked such a checkin unconditionally, writing its event row despite insertCheckinMlinks silently skipping the missing-blob F-card.

Because collectCrosslinkCandidates only reselects a rid with no event row, that half-linked checkin was permanently stranded: no future sweep, no matter how many sync rounds ran or when the missing blob eventually arrived, would ever revisit it. A clone that terminated before the blob arrived could leave the repository with a leaf checkin selected by CalcUpdateVersion whose Checkout.Update would fail with "blob not found" -- with no recovery path.

Factor the sweep's deferral bookkeeping (availability cache, deferred rids, missing blobs) into checkinDeferralGuard, shared by both linkState (the sweep) and cascadeLinker (the cascade). The cascade now checks shouldDefer before opening a checkin's savepoint, so a deferred checkin writes nothing to the batch -- matching the sweep's defer semantics exactly, and preserving the invariant both paths depend on: a deferred rid must carry no event row, or the sweep's own candidate query can never find it again.

deferCheckin's sweep-facing behavior, and linkBatch's call site, are unchanged -- it is now a thin wrapper over the shared guard. check-in: 36d61bd582 user: [email protected] tags: trunk

afterDephantomize crosslinked every artifact it discovered through crosslinkSingle, which opened its own write transaction per artifact and passed an explicit nil content cache into crosslinkCheckin. Both choices suit an isolated artifact and neither suits the cascade: one phantom fill unblocks hundreds or thousands of related artifacts that share delta bases and parent manifests, so the walk paid a transaction commit per artifact and re-expanded content a sibling had just expanded. A CPU profile of round 7 of a fossil-corpus clone showed exactly that pair -- _vdbeCommit plus expandChain/decompress reached through crosslinkSingle -- as the remaining 300+ seconds after the statement-cache fix.

crosslinkSingle had no callers outside the cascade, so it is replaced rather than kept alongside a second strategy. cascadeLinker buffers the artifacts the walk discovers and links them through linkArtifact -- the same type switch the whole-repository sweep uses, so the two paths no longer carry near-duplicate switches -- on one content.Cache for the whole cascade, committing once per crosslinkBatchSize artifacts instead of once per artifact. The sweep's own batching, cache, and tuning constants are untouched.

The walk's delta-child query filters on committed mlink rows, so deferring writes past it could change which artifacts it visits. Orphans are the only discovery that query can observe -- delta.rid is a primary key, so every other buffered artifact was itself produced by that query and cannot come back -- so the buffer flushes after an orphan batch and before the query, preserving the original order: orphans crosslinked, orphan rows cleared, children queried.

Failure handling keeps what per-artifact transactions bought. An artifact that cannot be expanded, parsed, or linked is logged with the same wording and skipped while the rest of the batch commits, and each artifact's writes are wrapped in a savepoint so a link that fails partway leaves no partial event/plink/mlink/tagxref behind. Cancellation is unchanged in the walk and newly observed inside a flush on the sweep's stride; an open batch is abandoned rather than committed, so committed batches stand and the next Crosslink sweep re-selects whatever still has no event row.

Tests: a characterization test pins the cascade's derived rows and visit count over a graph covering both sides of the mlink filter -- an orphan that writes mlink rows and one that does not, and so is still visited twice -- against the dump the per-artifact shape produced. A regression guard asserts one commit and one expansion per artifact with the parent manifest served from cache for a 300-artifact burst, plus tests for whole-batch flushing, savepoint isolation, and mid-flush cancellation. check-in: bfb2ad3aa4 user: [email protected] tags: trunk

AfterDephantomize crosslinks a just-filled phantom and walks the whole orphan/delta graph reachable from it. It runs synchronously mid-round as a clone stores received files (sync.storeReceivedFile), before the round returns and long before the post-round CrosslinkContext sweep that already observes the deadline. That walk took no context and was bounded only by a 1,000,000- iteration safety cap, so on a large graph a clone could run for many multiples of its configured deadline with the process busy inside this exact call chain and no cancellation ever reaching it (#166).

Thread context.Context through AfterDephantomize, afterDephantomize, and crosslinkSingle, and poll ctx.Done() inside the work-stack loop every crosslinkCancelCheckStride pops -- the same stride-batched check linkBatch already uses in the whole-repository sweep. On cancellation the walk returns what it has crosslinked so far, matching how the batch sweep abandons its remaining candidates. The clone path (cloneSession.handleFile) and server push handler (handler.handleFile) pass their real request context through; the general sync loop, whose deadline is not threaded to handleFileCard, passes context.Background() to keep its behavior unchanged.

This is an additive, disjoint cancellation path: the post-round batch sweep and its existing cancellation tests are untouched.

Regression test TestAfterDephantomizeObservesCancellationMidWalk builds a root-plus-fanout delta graph larger than the stride and drives the walk with a context that goes live -> cancelled mid-walk, asserting the cascade stops early instead of crosslinking every node. check-in: a633f335c9 user: [email protected] tags: trunk

insertCheckinMlinks walked only the child's F-cards, so two cases diverged from canonical Fossil's add_mlink:

1. A file present in the primary parent but dropped by OMISSION from a full (non-delta) child manifest got no mlink row at all. Canonical emits a delete row (fid=0) for it. A new second pass, insertOmittedDeletions, compares the primary parent's file set against the child's effective tree and emits a fid=0 delete row for every parent file the child no longer carries and named in no F-card.

2. A rename resolved its pid against the NEW path, which is absent from the parent, so pid defaulted to 0 as if the file were a fresh add -- losing pre-rename ancestry and defeating content_deltify. The F-card walk now resolves the parent lookup against the OLD path (OldName) for a rename, so pid is the parent's pre-rename blob rid.

Verified byte-for-byte against the fossil 2.28 binary (TestFossilBinaryMlinkParity_DeletionAndRename): canonical emits BOTH a rename row keyed on the new name (pfnid = old name) AND a delete row for the vacated old name, both carrying pid = the old-path blob rid. The omission pass reproduces that pair, and applyFileCard now vacates a rename's prior name from the effective tree so the old name is correctly seen as dropped in delta manifests too. check-in: 9fed05730c user: [email protected] tags: trunk

* manifest: run leaf/tag repair on interrupted crosslink sweep

crosslinkSweep skipped the repairLeafTable/repairTagPropagation pass whenever linkCandidatesInOrder returned an error, including a context cancellation. After #89's batched sweep, batches commit as they go, so a sweep cancelled after committing at least one batch left the plink edges and tagxref origins those batches wrote without their leaf/tag consequences -- stale until a later sweep linked something, and a later sweep whose remaining candidates all link zero artifacts skips the same gate and leaves the window open longer than one sweep.

Gate the repair on linked > 0 rather than on sweep success and return the sweep's own error after repairing, so an interrupted-but-committed sweep still recovers derived leaf/tag state on the way out. The repair runs in its own context-free transaction, so a cancelled ctx interrupts the candidate sweep but not the recovery, preserving the batched commit/rollback consistency guarantees.

Closes #143.

* manifest: join repair error with sweep error on interrupted sweep

When both the sweep is interrupted and the deferred leaf/tag repair pass fails, the repair error was dropped entirely -- only the sweep error was returned. A real repair failure (disk full, corruption, constraint violation) during a cancelled sweep would then persist the exact stale leaf/tag window this fix closes with no signal.

Join both with errors.Join so the repair failure is surfaced while errors.Is(err, context.Canceled) still holds for the three callers (clone.go, session.go, handler.go), all of which wrap with %w and none of which use exact equality. check-in: 7bc832f339 user: [email protected] tags: trunk

* manifest: batch crosslink, memoize availability, preload mlink parents (#89)

Three throughput fixes for the whole-repository crosslink sweep, none of which change its observable output. Profiling the sweep on the Fossil SCM corpus (67,503 blobs, 27,084 events) after the delta-chain-order work (#126) found the earlier bottlenecks had moved on:

- Per-candidate transactions made every SQL statement open its own WAL read-transaction, whose setup cost climbs as the uncheckpointed WAL grows -- throughput collapsed from ~200 events/sec to ~7 within a few seconds of a large sweep. The sweep now commits in batches of crosslinkBatchSize candidates so SQLite checkpoints and truncates the WAL between batches, holding read-transaction cost flat. Batch-granular durability is correct for derived data: an interrupted sweep leaves committed batches and the candidate query re-selects only what is still unlinked. CrosslinkContext is split into crosslinkSweep / collectCrosslinkCandidates / linkCandidatesInOrder / linkBatch, with the per-artifact dispatch pulled into linkArtifact and the deferral rollup into deferCheckin / logDeferredCheckins.

- missingCheckinRefs walked a blob's whole delta chain (content.IsAvailable) once per F-card of every check-in to decide deferral -- ~40% of the sweep, re-walking the same file chains over and over. content.AvailabilityCache memoizes the verdict per rid and stops a walk at the first already-decided ancestor, bounding total walk work by the number of distinct blobs rather than F-card references.

- resolveMlinkParent looked up each F-card's parent version with SELECT ... FROM mlink WHERE mid=? AND fnid=?, which SQLite answers via the fnid index -- returning a file's entire crosslinked history, so the lookup grew as the mlink table did. loadMlinkParents now reads each parent's file set once with a single mid-indexed scan and resolves every F-card from the in-memory map. It reads exactly the same rows, so the mlink output is unchanged.

The crosslink helpers now take the batch's *db.Tx directly (replacing r *repo.Repo + per-helper r.WithTx), matching the single-transaction shape of internal/verify.Rebuild and letting a whole batch commit atomically; tag application routes through tag.ApplyTagWithTx. updateAttachmentComments materializes its cursor before writing so a single connection never runs two statements at once. crosslinkSingle (dephantomize) wraps its one artifact in its own transaction.

Whole suite green (1235 tests, 35 packages, incl. dst and the fossil-binary comparison). This does not by itself land #89's "minutes not hours" target; the dominant remaining cost is a separate, pre-existing mlink over-insertion documented on the issue.

* manifest: diff crosslink mlink against the parent manifest (#89)

insertCheckinMlinks emitted one mlink row per F-card and read pmid/pid from the parent's mlink rows. On the Fossil SCM repository that is ~180x canonical's row count (a full sweep would write ~19M rows against fossil's own 104,880), and because delta-chain order (#102/#126) routinely crosslinks a child before its parent, the parent's mlink rows were usually absent, so ~97% of rows defaulted to pid=0 -- wrong parent-column data independent of the volume.

Rewrite it to diff each check-in's file list against its primary parent's expanded manifest -- the same diff canonical Fossil computes in add_mlink (src/manifest.c) and the rebuild path already does in internal/verify.rebuildMlinks -- and emit a row only for a file that was added, modified, deleted, or renamed. pmid/pid now come from the parent MANIFEST, which is always present once a check-in is eligible to link, so pid is correct regardless of crosslink order. The parent expansion runs through the sweep's shared content cache, so each parent's delta chain is walked at most once across all its children.

Measured on the Fossil SCM corpus: 127,775 mlink rows (canonical: 104,878) with pid populated on 74% of rows (was ~3%); full-repo crosslink completes in ~15 minutes rather than the prior ~hour. The residual gap is 95 PGP-clearsigned parent manifests deck.Parse cannot yet strip, whose ~98 children fall back to the old all-added behavior -- a pre-existing deck limitation tracked separately. check-in: 81fbc0be5f user: [email protected] tags: trunk

The buffer-reuse fix in #145/#150 cut the cost of expanding one delta chain once. It did nothing for callers that expand many versions of the same artifact in a single operation: each bare content.Expand still replayed the whole O(depth) chain, so a walk over N overlapping revisions cost O(N^2) blob reads even though content.Cache already existed for exactly this pattern.

Audited every non-test content.Expand caller. Two access patterns genuinely repeat over overlapping delta chains and are now routed through a per-operation content.Cache:

- annotate walks every revision of one file along the parent chain. content_deltify stores each older file blob as a delta against its successor, so the per-revision expansions are all walks of the same few chains. One cache threaded through the walk (file blob + manifest listing via new manifest.ListFilesCached/GetManifestCached, both nil-passthrough so existing callers are unchanged) turns it linear. Measured on a deltified 150-revision file: 1.60s -> 0.65s (2.5x), 764k -> 308k allocs; the win grows with history depth.

- verify.Verify and verify.Rebuild sweep every blob more than once (Rebuild: checkBlobs + rebuildManifests + rebuildTags x2). A single cache shared across the pass walks each chain once instead of once per sweep and amortizes overlap within each sweep. Measured on a 150-revision two-file repo: Rebuild 3.34s -> 0.068s (48x), 4.59M -> 98k allocs.

Callers that only ever touch the tip or make a single pass over distinct blobs (branch create, merge-config load, R-card / delta manifest build, MergeParentFiles, crosslinkSingle, shun undelta, pairwise Diff) are left on bare Expand, each with a comment stating why a cache cannot amortize their access.

No behavior change: expansion output is byte-identical, verified across the modernc and ncruces drivers with REQUIRE_FOSSIL_BIN=1 cross-checks and the DST suite. check-in: 0fd2a507b8 user: [email protected] tags: trunk

* fix: emit renames as a single F-card with the "w" placeholder (#51)

buildCommitFiles seeded its working set from the parent manifest and layered vfile changes on top, but never consulted vfile.origname. A pure rename has identical content, so ScanChanges clears its chnged flag; the new name then never entered the working set while the old name carried forward from the parent, so the manifest emitted the file's content under its old pathname and dropped the rename entirely.

Drive renames off vfile.origname, mirroring canonical Fossil's manifest query (src/checkin.c:1939): applyRenames retires the old pathname and records it on the surviving new-name entry, so a rename is emitted exactly once. The unchanged-content pass then loads the blob and preserves OldName.

Serialize the rename card to match stock Fossil: force a "w" permission placeholder when a renamed file's perm would otherwise be empty, keeping the prior-name field in its 4th positional slot (src/checkin.c:1999). Thread OldName through manifest.File and storeFileBlobs so the commit path can carry it.

Verified the exact card "F greet.txt <uuid> w hello.txt" against a real fossil binary for the same rename scenario.

* fix: never emit a renamed-then-deleted file in the commit manifest

applyRenames enumerated every vfile row with origname != pathname, including rows marked deleted, and unconditionally seeded a new-name entry for them. A file that was renamed and then unmanaged therefore committed as a rename F-card under its new name. The deleted-files pass could not catch it: it only knows the file's current pathname, so deleting the new name from a map keyed on parent-manifest pathnames was a no-op and nothing retired the old name.

Canonical Fossil excludes deleted rows from its manifest query outright (WHERE NOT deleted OR NOT is_selected). Match that: a deleted renamed row now retires both names and emits nothing, keeping the existing guard that leaves the old name alone when a live vfile row has reclaimed it.

Also move the deck marshaller's prior-name field inside the UUID guard. The field is positional and only meaningful after a hash; a deletion card carrying one would serialize as "F name oldname", with the old name misread as the UUID.

Tests: rename-then-delete emits no card under either name; a rename chain that recycles a freed pathname emits both cards with the right prior names and content. check-in: 29c2c5c6f5 user: [email protected] tags: trunk

The clone command hardcoded a 10-minute context deadline with no way to raise or disable it, and the deadline did not reliably interrupt a slow clone.

- RepoCloneCmd gains a --timeout flag (default 10m, matching the old behavior; 0 disables the deadline for very large repositories). cloneContext branches away from context.WithTimeout for a non-positive timeout -- WithTimeout(parent, 0) yields an already-expired deadline, which would abort instantly the very clone the flag exists to unbound.

- The whole-repository crosslink sweep now observes context cancellation. It runs after the exchange-round loop, so it had no round boundary of its own at which the deadline could fire -- on a large repository it could run long past the deadline uninterrupted. manifest.CrosslinkContext checks the context every 256 candidates; manifest.Crosslink stays a background-context wrapper so its ~30 existing callers are unchanged. processResponse gains the same batched check so a large single-round batch is interruptible too, and the sync session's and the serve handler's post-convergence sweeps are wired to their contexts for the same reason.

Each cancellation point is pinned by a test that fails without it: the crosslink and processResponse tests drive a context that survives the first poll and dies before the second, so only the batched check can observe it; the crosslink-sweep test cancels after a zero-card final round, so nothing before the sweep is left to notice.

Also adds a clone -> serve -> reclone fidelity regression guard (#114), built with the canonical fossil binary rather than this library's Commit. Commit does not deltify, so a corpus built through it holds zero delta rows and never reaches the serve side's deltified-row expansion branch -- a green result from such a corpus proves nothing about #114. The guard now asserts the source holds delta rows before measuring anything, and compares the second-generation clone's tip, artifact count, blob-store size, delta-row count and checked-out tree content against the source.

Measured against that corpus (71 artifacts, 54 delta rows), #114 does not reproduce: gen1 and gen2 both hold the same 54 delta rows, the same tip and artifact count, a 1.009 store ratio, identical checked-out content, and pass fossil test-integrity. The receive path re-deltifies what the serve side expanded, via content.Deltify on the crosslink mlink path, so the expansion the serve side performs does not survive into the store. check-in: a02c9572da user: [email protected] tags: trunk

* Wire manifest.AfterDephantomize into the phantom fill path

AfterDephantomize was a faithful, tested port of Fossil's after_dephantomize (content.c:389-456) that nothing in production ever called — it crosslinked correctly in isolation but sat disconnected from the real receive path.

storeResolvedContent fills a phantom row in place when the full content for a previously-unknown UUID arrives (matching canonical's content_put_ex filling a phantom at content.c:626), but stopped short of crosslinking it or any delta children waiting on it. Return the filled rid from storeResolvedContent and call AfterDephantomize on it from storeReceivedFile once the fill transaction has committed — outside the transaction, since AfterDephantomize opens its own connection via repo.Repo.DB() and calling it from inside the tx's closure would contend with the still-open write lock.

Added an end-to-end regression test that delivers a checkin manifest as a delta against a base that hasn't arrived yet, then delivers the base, and asserts the manifest is crosslinked (its event row exists) immediately — with no explicit call to manifest.Crosslink anywhere in the test. This proves AfterDephantomize fires on the real fill path, not only under its own unit tests.

* Log crosslink failures during AfterDephantomize instead of swallowing them

A crosslink error while filling a phantom blob (or one of its orphan/delta dependents) previously vanished silently, leaving storeReceivedFile report success with the artifact filled but un-crosslinked. Surface it via slog.Warn so real corruption on the fill path is at least visible, matching the non-blocking nit from review. check-in: 94919ca357 user: [email protected] tags: trunk

* fix: zero-length file commit panic and stale missing-file carry-forward

Two related bugs in the checkin path:

#68 - Committing a zero-length file panicked in blob.Store, which treated an empty artifact (a normal, well-known Fossil blob) as invalid input. Worse, manifest.Checkin's postcondition defer fired unconditionally during the panic's stack unwind and re-panicked with its own unrelated "manifestRid must be positive" message, masking the real cause. Fixed both: blob.Store now accepts zero-length (and nil) content, and Checkin's defer re-panics with the original panic value instead of asserting a postcondition that was never going to hold mid-unwind. Fixing the panic surfaced a second bug in the same family: buildCommitFiles used len(Content) > 0 to detect "already loaded from disk," which can't distinguish a genuinely empty file from one not yet loaded, so a newly added empty file failed with "no RID for unchanged file." Replaced the length sentinel with an explicit loaded-set.

#79 - A tracked file deleted from disk without Unmanage/Remove was silently carried forward into the new manifest with its last-committed content, rather than being detected. Verified against fossil 2.28: canonical fails a commit when a missing tracked file is in scope (either the implicit "commit everything" case, or explicitly named), but leaves an out-of-scope missing file alone. buildCommitFiles now stats every in-scope tracked file before treating it as unchanged and errors clearly, naming the file, when it's gone; a file outside the commit's scope (relevant only to an explicit Enqueue) is left untouched exactly as before. Updated the two existing tests that had documented the old silent-carry-forward behavior as deliberately deferred to this issue.

* fix: add CHANGELOG entry and unwrap-safe stat checks for #68/#79

Adds the CHANGELOG entries the #79 acceptance criteria required for a user-visible behavior change (missing tracked file now hard-fails a commit instead of silently carrying stale content forward) and for the #68 zero-length-file panic fix.

Also switches checkin.go's two os.IsNotExist(err) checks to errors.Is(err, os.ErrNotExist), which correctly unwraps the %w-wrapped os.ErrNotExist that MemStorage.Stat returns. Production was already correct (OSStorage.Stat returns a raw *PathError), but the tests were only exercising a generic fallback error branch instead of the intended "tracked file missing from disk" message. check-in: 0351167ae5 user: [email protected] tags: trunk