Timeline

* chore: bump driver submodule requires, drop root replace directives

The db/driver submodules now have real tags under the corrected go-libfossil module path (v0.3.0 for modernc and ncruces). Root no longer needs to resolve them from local directories via replace.

go mod tidy also cleared the stale old-path (danmestas/libfossil) lines the earlier replace bridge left behind in go.sum.

* chore: drop submodule replace directives, pin root require to a pseudo-version

The three submodules required the root module (which they import directly or transitively) but no root tag has ever existed under the corrected go-libfossil path -- every prior tag (v0.1.0 through v0.6.3) still declares the old danmestas/libfossil path in its own go.mod, so pinning to any of them would reproduce the exact declares-as-X-required-as-Y failure the module rename was fixing.

Pinned each submodule's require on root to a pseudo-version of the just-pushed go.mod cleanup commit (48111c1) instead of a tag -- the standard resolution for this bootstrapping case. This is a stopgap: once the next root release tag exists, these can be bumped to point at it directly instead of the pseudo-version, but there's no functional difference in the meantime.

go mod tidy on each submodule also dropped now-unneeded indirect requires (modernc/ncruces don't actually import each other; the requires existed only to satisfy the local replace directives) and cleared the stale old-path go.sum lines.

All four go.mod files are now free of replace directives. check-in: 2de18e7747 user: [email protected] tags: trunk

insertCheckinMlinks walked only the child's F-cards, so two cases diverged from canonical Fossil's add_mlink:

1. A file present in the primary parent but dropped by OMISSION from a full (non-delta) child manifest got no mlink row at all. Canonical emits a delete row (fid=0) for it. A new second pass, insertOmittedDeletions, compares the primary parent's file set against the child's effective tree and emits a fid=0 delete row for every parent file the child no longer carries and named in no F-card.

2. A rename resolved its pid against the NEW path, which is absent from the parent, so pid defaulted to 0 as if the file were a fresh add -- losing pre-rename ancestry and defeating content_deltify. The F-card walk now resolves the parent lookup against the OLD path (OldName) for a rename, so pid is the parent's pre-rename blob rid.

Verified byte-for-byte against the fossil 2.28 binary (TestFossilBinaryMlinkParity_DeletionAndRename): canonical emits BOTH a rename row keyed on the new name (pfnid = old name) AND a delete row for the vacated old name, both carrying pid = the old-path blob rid. The omission pass reproduces that pair, and applyFileCard now vacates a rename's prior name from the effective tree so the old name is correctly seen as dropped in delta manifests too. check-in: 9fed05730c user: [email protected] tags: trunk

VerifyZ validated a control artifact by hashing everything up to the last 35 bytes and comparing against the Z card it expected to find there. When a manifest arrived PGP-clearsigned, the raw blob carried a trailing "-----BEGIN PGP SIGNATURE-----" ... "-----END PGP SIGNATURE-----" block after the Z card, so the last-35-byte window landed inside the signature framing instead of on the Z card, and verification failed. deck.Parse propagated that failure, so ~95 of Fossil's own 2007-era clearsigned check-ins could not be parsed -- and any child whose primary parent was one of them lost its parent's file tree in loadPrimaryParentFiles, cascading into all-added mlink data (~98 affected children on the real corpus).

Mirror fossil's src/manifest.c remove_pgp_signature: strip the armor header (up to the blank line that ends it) and the trailing signature block, isolating the inner content the Z card is actually computed over, then verify and parse that. The stripping is an identity on non-clearsigned input (the common case) and idempotent.

The artifact's content-addressed hash is taken over the raw blob elsewhere and is untouched by this change: a clearsigned manifest still resolves to the same UUID it always has. Verified against the real Fossil check-in 61829b07 (aku, 2007-11-29), whose raw-blob SHA-1 still equals its UUID and whose inner Z-card MD5 now validates; cross-checked byte-for-byte against fossil 2.28. check-in: c1cbd26795 user: [email protected] tags: trunk, observer/otel/v0.2.0, db/driver/ncruces/v0.3.0, db/driver/modernc/v0.3.0

* manifest: run leaf/tag repair on interrupted crosslink sweep

crosslinkSweep skipped the repairLeafTable/repairTagPropagation pass whenever linkCandidatesInOrder returned an error, including a context cancellation. After #89's batched sweep, batches commit as they go, so a sweep cancelled after committing at least one batch left the plink edges and tagxref origins those batches wrote without their leaf/tag consequences -- stale until a later sweep linked something, and a later sweep whose remaining candidates all link zero artifacts skips the same gate and leaves the window open longer than one sweep.

Gate the repair on linked > 0 rather than on sweep success and return the sweep's own error after repairing, so an interrupted-but-committed sweep still recovers derived leaf/tag state on the way out. The repair runs in its own context-free transaction, so a cancelled ctx interrupts the candidate sweep but not the recovery, preserving the batched commit/rollback consistency guarantees.

Closes #143.

* manifest: join repair error with sweep error on interrupted sweep

When both the sweep is interrupted and the deferred leaf/tag repair pass fails, the repair error was dropped entirely -- only the sweep error was returned. A real repair failure (disk full, corruption, constraint violation) during a cancelled sweep would then persist the exact stale leaf/tag window this fix closes with no signal.

Join both with errors.Join so the repair failure is surfaced while errors.Is(err, context.Canceled) still holds for the three callers (clone.go, session.go, handler.go), all of which wrap with %w and none of which use exact equality. check-in: 7bc832f339 user: [email protected] tags: trunk

* manifest: batch crosslink, memoize availability, preload mlink parents (#89)

Three throughput fixes for the whole-repository crosslink sweep, none of which change its observable output. Profiling the sweep on the Fossil SCM corpus (67,503 blobs, 27,084 events) after the delta-chain-order work (#126) found the earlier bottlenecks had moved on:

- Per-candidate transactions made every SQL statement open its own WAL read-transaction, whose setup cost climbs as the uncheckpointed WAL grows -- throughput collapsed from ~200 events/sec to ~7 within a few seconds of a large sweep. The sweep now commits in batches of crosslinkBatchSize candidates so SQLite checkpoints and truncates the WAL between batches, holding read-transaction cost flat. Batch-granular durability is correct for derived data: an interrupted sweep leaves committed batches and the candidate query re-selects only what is still unlinked. CrosslinkContext is split into crosslinkSweep / collectCrosslinkCandidates / linkCandidatesInOrder / linkBatch, with the per-artifact dispatch pulled into linkArtifact and the deferral rollup into deferCheckin / logDeferredCheckins.

- missingCheckinRefs walked a blob's whole delta chain (content.IsAvailable) once per F-card of every check-in to decide deferral -- ~40% of the sweep, re-walking the same file chains over and over. content.AvailabilityCache memoizes the verdict per rid and stops a walk at the first already-decided ancestor, bounding total walk work by the number of distinct blobs rather than F-card references.

- resolveMlinkParent looked up each F-card's parent version with SELECT ... FROM mlink WHERE mid=? AND fnid=?, which SQLite answers via the fnid index -- returning a file's entire crosslinked history, so the lookup grew as the mlink table did. loadMlinkParents now reads each parent's file set once with a single mid-indexed scan and resolves every F-card from the in-memory map. It reads exactly the same rows, so the mlink output is unchanged.

The crosslink helpers now take the batch's *db.Tx directly (replacing r *repo.Repo + per-helper r.WithTx), matching the single-transaction shape of internal/verify.Rebuild and letting a whole batch commit atomically; tag application routes through tag.ApplyTagWithTx. updateAttachmentComments materializes its cursor before writing so a single connection never runs two statements at once. crosslinkSingle (dephantomize) wraps its one artifact in its own transaction.

Whole suite green (1235 tests, 35 packages, incl. dst and the fossil-binary comparison). This does not by itself land #89's "minutes not hours" target; the dominant remaining cost is a separate, pre-existing mlink over-insertion documented on the issue.

* manifest: diff crosslink mlink against the parent manifest (#89)

insertCheckinMlinks emitted one mlink row per F-card and read pmid/pid from the parent's mlink rows. On the Fossil SCM repository that is ~180x canonical's row count (a full sweep would write ~19M rows against fossil's own 104,880), and because delta-chain order (#102/#126) routinely crosslinks a child before its parent, the parent's mlink rows were usually absent, so ~97% of rows defaulted to pid=0 -- wrong parent-column data independent of the volume.

Rewrite it to diff each check-in's file list against its primary parent's expanded manifest -- the same diff canonical Fossil computes in add_mlink (src/manifest.c) and the rebuild path already does in internal/verify.rebuildMlinks -- and emit a row only for a file that was added, modified, deleted, or renamed. pmid/pid now come from the parent MANIFEST, which is always present once a check-in is eligible to link, so pid is correct regardless of crosslink order. The parent expansion runs through the sweep's shared content cache, so each parent's delta chain is walked at most once across all its children.

Measured on the Fossil SCM corpus: 127,775 mlink rows (canonical: 104,878) with pid populated on 74% of rows (was ~3%); full-repo crosslink completes in ~15 minutes rather than the prior ~hour. The residual gap is 95 PGP-clearsigned parent manifests deck.Parse cannot yet strip, whose ~98 children fall back to the old all-added behavior -- a pre-existing deck limitation tracked separately. check-in: 81fbc0be5f user: [email protected] tags: trunk

encodeCFile emitted the cfile payload as bare zlib. Fossil's on-disk cfile format, produced by blob_compress() in src/blob.c, is a 4-byte big-endian uncompressed-size prefix followed by the zlib stream, and fossil's receiver (content_put_ex) stores the wire payload verbatim into blob.content, later reading it back through blob_uncompress(), which reads that prefix to size its output before inflating. Without the prefix a real fossil client that clones from us stores every full (non-delta) artifact unreadable: fossil clone exits 0 but rebuild lands zero check-ins and manifests decode to garbage.

Prepend the 4-byte prefix in encodeCFile and count it in CSIZE, so the payload matches blob_compress byte-for-byte. The prefix value is len(Content) (the bytes being compressed -- a delta payload for delta cards), which stays distinct from the USIZE header field (the full reconstructed size). Our internal storage already used fossil's framing (blob.Compress), and the decoder already accepted both framings and rebuilt StoredBlob, so this is a wire-encode-only fix; receive and on-disk paths are unchanged.

TestCloneRealFossilWithDeltaChain now asserts six materialized check-ins instead of skipping against this issue, and a new TestCloneRealFossilFullContent drives a delta-free corpus through a real fossil clone+rebuild, asserting the check-in count and that a cloned manifest decodes to parseable manifest cards. check-in: 00e69bc9e0 user: [email protected] tags: trunk

* sync: correct the clone cfile comment to match the actual code path (#140)

The comment claimed clone "retransmits the repository's stored compressed content verbatim... blob.Load returns exactly that." That is not what the code does: blob.Load decompresses the stored blob and encodeCFile then re-compresses it with zlib before it goes on the wire. The wire result is correct because the decompress/re-compress round-trip is byte-deterministic, but nothing forwards the stored compressed bytes untouched. Correct the comment to describe the real mechanism (decompress via blob.Load, re-compress via encodeCFile) and state explicitly why the round-trip is still stable.

Documentation-only; no behavior change.

* sync: retransmit deltas during clone instead of expanding them (#141)

The clone send path expanded every deltified row to full content before putting it on the wire, giving up the delta-transmission bandwidth the xfer protocol is designed for. Restore delta transmission on the clone send path.

Send ordering. content.Deltify deltifies the OLDER artifact against the NEWER one, so a delta's source almost always has a greater rid than the delta and would not yet have been sent under the ascending-rid clone cursor. Rather than reorder the whole cursor, buildCloneArtifact walks each artifact's delta chain toward its full-content tip and emits the chain source-first, mirroring canonical fossil's recursive send_file: the tip (or the lowest still-sendable anchor) leads, each delta follows the source it names, and the requested rid is last. A per-round "sent" set skips any source a dependent's chain already pulled in ahead of its ascending position, matching fossil's per-request bag. The snapshot bound (#17) is preserved: a delta whose source lies outside the clone snapshot, is a phantom, or is private under a public clone degrades to full content so no out-of-snapshot row is ever leaked.

Wire format. A delta goes out as an uncompressed "file UUID DELTASRC SIZE" card, exactly as canonical fossil's send_delta_native does, not as a "cfile". Full content still rides a compressed cfile, preserving the #98/#113 wire-size win.

Scope. This is a bandwidth win, verified content-identical for libfossil<->libfossil clones by the self-round-trip tests, with the source-before-dependent invariant guarded by TestEmitCloneBatchSourcePrecedesDelta. The receive side already re-deltifies via content.Deltify on the crosslink path (#114), so store and tip fidelity are unchanged. Measured on a 120-artifact, 59-delta corpus, a full clone's reply bytes drop from 32,551 (expand-before-send) to 17,419, a 46% reduction, close to the 14,941-byte stored size.

This does NOT make a real fossil client's clone usable. Full content still rides a compressed cfile, which go-libfossil emits as bare zlib while fossil expects [4-byte size][zlib] framing, so a real fossil client decodes full content to garbage and rebuilds to zero check-ins. That is a separate, pre-existing bug (present on unmodified main), tracked as #152. TestCloneRealFossilWithDeltaChain asserts real usability and skips against

* test(sync): update real-fossil delta test doc to the #141 mechanism

* test(sync): prove clone usability, drop false real-fossil interop claim (#141)

TestCloneRealFossilWithDeltaChain asserted only that `fossil clone` exits 0, which a real fossil client does even when the clone is empty and its content is undecodable. Strengthen it to prove usability: rebuild the clone and count its check-ins. The count is zero -- a real fossil client decodes go-libfossil's full-content cfile payload (bare zlib) to garbage because fossil's on-disk cfile format is [4-byte size][zlib] -- so the test skips against the tracking issue (#152) instead of silently passing on corrupt content. When #152 lands the clone materializes all six check-ins and the test passes for real.

Also correct the doc comment and the handler.go send-path comment, which claimed #141's send-order + uncompressed-delta-card change keeps a real fossil 2.28 client's rebuild from failing. It does not: that change is a libfossil<->libfossil bandwidth win (verified content-identical by the self-round-trip tests); real-fossil interop for delta-bearing clones remains blocked on the separate cfile-framing bug (#152), present on unmodified main. check-in: 5f66319ea3 user: [email protected] tags: trunk

The buffer-reuse fix in #145/#150 cut the cost of expanding one delta chain once. It did nothing for callers that expand many versions of the same artifact in a single operation: each bare content.Expand still replayed the whole O(depth) chain, so a walk over N overlapping revisions cost O(N^2) blob reads even though content.Cache already existed for exactly this pattern.

Audited every non-test content.Expand caller. Two access patterns genuinely repeat over overlapping delta chains and are now routed through a per-operation content.Cache:

- annotate walks every revision of one file along the parent chain. content_deltify stores each older file blob as a delta against its successor, so the per-revision expansions are all walks of the same few chains. One cache threaded through the walk (file blob + manifest listing via new manifest.ListFilesCached/GetManifestCached, both nil-passthrough so existing callers are unchanged) turns it linear. Measured on a deltified 150-revision file: 1.60s -> 0.65s (2.5x), 764k -> 308k allocs; the win grows with history depth.

- verify.Verify and verify.Rebuild sweep every blob more than once (Rebuild: checkBlobs + rebuildManifests + rebuildTags x2). A single cache shared across the pass walks each chain once instead of once per sweep and amortizes overlap within each sweep. Measured on a 150-revision two-file repo: Rebuild 3.34s -> 0.068s (48x), 4.59M -> 98k allocs.

Callers that only ever touch the tip or make a single pass over distinct blobs (branch create, merge-config load, R-card / delta manifest build, MergeParentFiles, crosslinkSingle, shun undelta, pairwise Diff) are left on bare Expand, each with a comment stating why a cache cannot amortize their access.

No behavior change: expansion output is byte-identical, verified across the modernc and ncruces drivers with REQUIRE_FOSSIL_BIN=1 cross-checks and the DST suite. check-in: 0fd2a507b8 user: [email protected] tags: trunk

delta.Apply's one-time capacity upgrade (#127) only engages above the 64 KiB threshold, so it is a no-op for the typical source file under 64 KiB. But below the threshold Apply already sizes its output buffer exactly, so there was never a 5x overhead there to remove -- extending the threshold buys nothing. The real per-link cost of a chain walk was elsewhere, and a profile of a plain Expand of a deep sub-64 KiB chain shows it: a fresh ~32 KiB zlib history window allocated per delta load (~27% of bytes), one full-size Apply output buffer per link (~16%), and a fresh inflate buffer per link. All three are O(depth) allocations for a single cold Expand, which is O(depth), making a full-history walk O(n^2).

Attack the per-link constant inside the one choke point every expansion goes through, content.expandChain, leaving the on-disk chain shape untouched:

- delta.ApplyInto(dst, source, delta) reconstructs into a caller-supplied buffer, reusing its capacity. Apply is now ApplyInto(nil, ...), unchanged. A reused buffer's capacity is proven prior work, not a wire-header claim, so it is a safe starting size; the anti-DoS growth path (growOutputCap, keyed off len(delta) not the targetLen header) is preserved for buffers a target outgrows. - blob.Inflater loads a run of blobs through one zlib reader, reset per blob instead of rebuilt, paying the ~32 KiB window setup once per chain. Each Load still returns a freshly owned buffer, so no returned-slice-aliasing contract leaks to callers. - expandChain's plain-Expand replay (put == nil) ping-pongs two output buffers -- each link's target reuses the buffer from two links back, whose content was already consumed as the previous link's source, so dst and source never alias. The caching path (put != nil) is unchanged: put takes ownership of every materialized buffer, so those cannot be reused underneath it, but it still shares the reused zlib reader for its delta loads.

Measured on Apple M4 Pro, BenchmarkExpandOldestVersion, median of 3, 40x:

sub-64 KiB (~26 KB file), Expand of the oldest revision: depth 11: 883 KB -> 232 KB/op (3.8x), 1.12 ms -> 0.90 ms depth 49: 3.57 MB -> 341 KB/op (10.5x), 4.55 ms -> 2.03 ms depth 199: 14.2 MB -> 772 KB/op (18.4x), 14.84 ms -> 7.25 ms

>64 KiB (~206 KB file), no regression -- ping-pong + reader reuse help it too: depth 11: 4.11 MB -> 1.27 MB/op (3.2x), 3.40 ms -> 2.74 ms depth 49: 16.5 MB -> 1.81 MB/op (9.1x), 13.02 ms -> 7.69 ms depth 199: 65.2 MB -> 3.92 MB/op (16.6x), 46.3 ms -> 26.6 ms

Sub-64 KiB allocation is now near-flat in depth instead of linear in depth*targetLen. expand_reuse_test.go guards the reuse; delta and blob gain allocation-reduction unit tests. check-in: a2bcca2535 user: [email protected] tags: trunk

A stock fossil 2.28 client cloning a go-libfossil server transferred ~2.06x the repository's stored size, while the server's own response bodies measure 1.04x and a fossil-serving-fossil control measures 1.03x. The gap was a full second content-transfer pass, not a card-encoding problem.

Root cause is card order. Canonical's client re-issues `clone 3 SEQNO` every round it sees a server `push` card while its clone cursor is still positive (xfer.c:2706), and it learns the cursor reached zero only from the clone_seqno card. Canonical's server therefore emits the batch, then clone_seqno (xfer.c:1571), then push (xfer.c:1577). go-libfossil emitted push *first*, so the client queued one more clone request before it processed the terminal clone_seqno 0 — and the server served the entire repository a second time.

Relocate the clone-mode push-card emission to after processDataCards so it trails the batch's clone_seqno card, matching canonical's order. The clone now completes in a single content pass: re-measured end-to-end at 1.03x stored, identical to the fossil control (1.00x ratio). check-in: 1fd6c06a65 user: [email protected] tags: trunk

docs/site was the one module in the tree with no CI compile gate. It has its own go.mod but zero Go source of its own -- the module exists only to import the hextra theme via Hugo Modules -- so `go build ./...` there matches no packages and gates nothing (and `go vet ./...` fails on "no packages to vet"). The equivalent compile gate for a Hugo module is a Hugo build: it resolves the module import, compiles templates, and renders content, failing loud on a broken import or template.

Add a test-docs job that installs pinned Hugo Extended and runs `hugo --gc --minify` in docs/site, matching how the driver/observer submodules each get their own fail-loud step. check-in: be89e6397b user: [email protected] tags: trunk

* fix: emit renames as a single F-card with the "w" placeholder (#51)

buildCommitFiles seeded its working set from the parent manifest and layered vfile changes on top, but never consulted vfile.origname. A pure rename has identical content, so ScanChanges clears its chnged flag; the new name then never entered the working set while the old name carried forward from the parent, so the manifest emitted the file's content under its old pathname and dropped the rename entirely.

Drive renames off vfile.origname, mirroring canonical Fossil's manifest query (src/checkin.c:1939): applyRenames retires the old pathname and records it on the surviving new-name entry, so a rename is emitted exactly once. The unchanged-content pass then loads the blob and preserves OldName.

Serialize the rename card to match stock Fossil: force a "w" permission placeholder when a renamed file's perm would otherwise be empty, keeping the prior-name field in its 4th positional slot (src/checkin.c:1999). Thread OldName through manifest.File and storeFileBlobs so the commit path can carry it.

Verified the exact card "F greet.txt <uuid> w hello.txt" against a real fossil binary for the same rename scenario.

* fix: never emit a renamed-then-deleted file in the commit manifest

applyRenames enumerated every vfile row with origname != pathname, including rows marked deleted, and unconditionally seeded a new-name entry for them. A file that was renamed and then unmanaged therefore committed as a rename F-card under its new name. The deleted-files pass could not catch it: it only knows the file's current pathname, so deleting the new name from a map keyed on parent-manifest pathnames was a no-op and nothing retired the old name.

Canonical Fossil excludes deleted rows from its manifest query outright (WHERE NOT deleted OR NOT is_selected). Match that: a deleted renamed row now retires both names and emits nothing, keeping the existing guard that leaves the old name alone when a live vfile row has reclaimed it.

Also move the deck marshaller's prior-name field inside the UUID guard. The field is positional and only meaningful after a hash; a deletion card carrying one would serialize as "F name oldname", with the old name misread as the UUID.

Tests: rename-then-delete emits no card under either name; a rename chain that recycles a freed pathname emits both cards with the right prior names and content. check-in: 29c2c5c6f5 user: [email protected] tags: trunk

* content: add delta-chain depth benchmark for #85 re-measurement

BenchmarkExpandOldestVersion expands the oldest version of a ~26 KB, 400-line file at 12/50/200 revisions, mirroring the table in #85. The chain is built through content.Deltify so the on-disk shape matches what content_deltify produces: newest version whole, each older version a delta against the one newer.

Reports ns/op plus B/op and allocs/op so the per-link allocation profile the issue is about is directly observable.

* content: straddle delta.Apply's 64 KiB cap in the chain-depth benchmark

BenchmarkExpandOldestVersion measured a ~26 KB file, which sits below delta.Apply's deltaInitialCap: for targets that small the output buffer is allocated at exactly targetLen and the capped/growOutputCap path is never entered, so the arm cannot observe #81/#82 at all.

Parameterise the chain builder on file size and add a ~206 KB arm above the cap, so the two arms isolate the threshold rather than leaving it inferred from reading delta.go. Also move the chain-depth metric after the loop -- ResetTimer deletes user-reported metrics, so the depth reported before it never reached the output.

Refs #85

---------

Co-authored-by: Dan Mestas <[email protected]> check-in: c3338c4d38 user: [email protected] tags: trunk

The clone command hardcoded a 10-minute context deadline with no way to raise or disable it, and the deadline did not reliably interrupt a slow clone.

- RepoCloneCmd gains a --timeout flag (default 10m, matching the old behavior; 0 disables the deadline for very large repositories). cloneContext branches away from context.WithTimeout for a non-positive timeout -- WithTimeout(parent, 0) yields an already-expired deadline, which would abort instantly the very clone the flag exists to unbound.

- The whole-repository crosslink sweep now observes context cancellation. It runs after the exchange-round loop, so it had no round boundary of its own at which the deadline could fire -- on a large repository it could run long past the deadline uninterrupted. manifest.CrosslinkContext checks the context every 256 candidates; manifest.Crosslink stays a background-context wrapper so its ~30 existing callers are unchanged. processResponse gains the same batched check so a large single-round batch is interruptible too, and the sync session's and the serve handler's post-convergence sweeps are wired to their contexts for the same reason.

Each cancellation point is pinned by a test that fails without it: the crosslink and processResponse tests drive a context that survives the first poll and dies before the second, so only the batched check can observe it; the crosslink-sweep test cancels after a zero-card final round, so nothing before the sweep is left to notice.

Also adds a clone -> serve -> reclone fidelity regression guard (#114), built with the canonical fossil binary rather than this library's Commit. Commit does not deltify, so a corpus built through it holds zero delta rows and never reaches the serve side's deltified-row expansion branch -- a green result from such a corpus proves nothing about #114. The guard now asserts the source holds delta rows before measuring anything, and compares the second-generation clone's tip, artifact count, blob-store size, delta-row count and checked-out tree content against the source.

Measured against that corpus (71 artifacts, 54 delta rows), #114 does not reproduce: gen1 and gen2 both hold the same 54 delta rows, the same tip and artifact count, a 1.009 store ratio, identical checked-out content, and pass fossil test-integrity. The receive path re-deltifies what the serve side expanded, via content.Deltify on the crosslink mlink path, so the expansion the serve side performs does not survive into the store. check-in: a02c9572da user: [email protected] tags: trunk

Adds a wire-efficiency regression guard the issue flagged as missing: correctness-only round-trip tests never asserted card type or wire cost, so the file-card-instead-of-cfile amplification #98 fixed could silently return. TestCloneServesCompressedNotExpandedContent serves a small compressible corpus, clones it over the real HTTP transport, and fails if the server emits any expanded file card or if served bytes exceed 3x the repository's stored size. Card type is the load-bearing signal: the reply container is compressed, so a raw-file-card regression on compressible content hides inside a byte ratio -- the 3x bound is only a coarse backstop.

Also adds a WIRE_MEASURE-gated manual harness that re-measures serve-side amplification against a real fossil client through a byte-counting proxy, with a fossil-serving-fossil control. Current main: 2.06x stored end to end (server response bodies alone are 1.04x, matching the fossil control's 1.03x), down from the 17.9x this issue reported. check-in: 1a73119ed4 user: [email protected] tags: trunk

FossilEncode/FossilDecode implemented only 3 of the 8 escapes the artifact/sync format §4.3 mandates (\s, \n, \\). A field value carrying a real TAB, CR, VT, FF or NUL byte -- as canonical fossil writes it -- was mis-decoded and no longer re-marshalled to its source wire bytes, changing the artifact hash. Unrecognized escapes were also left literal instead of dropping the backslash per spec.

Add the remaining escapes (\t \r \v \f \0) to both directions and make an unrecognized escape drop its backslash, matching canonical fossil fossilize/defossilize (verified against fossil 2.28). Every caller -- A/C/H/L/J/F/T/U cards and the xfer cards -- shares these functions and gets the fix.

Closes #119 check-in: 1bfa6b2094 user: [email protected] tags: trunk

The test step runs `go test $(go list ./... | grep -v '/dst')`. When a package fails to compile, `go list` prints an error to stderr and omits that package from stdout, so it silently drops out of the test run rather than failing CI. `go build ./cmd/libfossil/` only covered the CLI binary's dependency closure, which missed internal/deck when it broke.

Add `go build ./...` ahead of the test step on both driver-tag legs (modernc and test_ncruces) so any package that fails to compile is a loud, fail-fast build error. This subsumes the narrower cmd/libfossil build, so that step is removed rather than kept alongside. check-in: fa558f254d user: [email protected] tags: trunk

The xfer HTTP server let net/http frame large clone/sync replies as Transfer-Encoding: chunked with no Content-Length. Release-tagged fossil <=2.23 reads the reply length only from Content-Length (src/http.c leaves iLength negative otherwise) and aborts with "server did not reply" on any chunked reply above ~256KB, so those clients could not clone a repository larger than that from a libfossil server.

writeXferResponse already materialises the whole response via Encode before the first write, so the exact body length is known for free. Set Content-Length explicitly; net/http then never falls back to chunked.

The fix is unconditional rather than negotiated. The triage proposed sniffing a client capability and keeping chunked for capable clients to avoid the buffering cost of Content-Length. That premise does not hold here: the server never streams -- Encode always returns a complete slice -- so chunked saves no memory over Content-Length, and #88's per-clone bound (DefaultCloneBatchBytes) still caps each reply either way. Content-Length is exactly what canonical fossil's own server emits, so it is strictly the more compatible framing for every client, including builds whose version banner cannot be trusted (#86).

Tests: a reconstructed <=2.23 client that requires a non-negative Content-Length and rejects chunked framing on a >256KB clone response (the discriminating test, since real fossil 2.28 tolerates chunked and cannot reproduce the failure), plus a modern-client no-regression clone of the same corpus. The existing real-fossil multi-batch clone interop test continues to pass unchanged.

Fixes #101 check-in: b3139353e1 user: [email protected] tags: trunk

* Wire manifest.AfterDephantomize into the phantom fill path

AfterDephantomize was a faithful, tested port of Fossil's after_dephantomize (content.c:389-456) that nothing in production ever called — it crosslinked correctly in isolation but sat disconnected from the real receive path.

storeResolvedContent fills a phantom row in place when the full content for a previously-unknown UUID arrives (matching canonical's content_put_ex filling a phantom at content.c:626), but stopped short of crosslinking it or any delta children waiting on it. Return the filled rid from storeResolvedContent and call AfterDephantomize on it from storeReceivedFile once the fill transaction has committed — outside the transaction, since AfterDephantomize opens its own connection via repo.Repo.DB() and calling it from inside the tx's closure would contend with the still-open write lock.

Added an end-to-end regression test that delivers a checkin manifest as a delta against a base that hasn't arrived yet, then delivers the base, and asserts the manifest is crosslinked (its event row exists) immediately — with no explicit call to manifest.Crosslink anywhere in the test. This proves AfterDephantomize fires on the real fill path, not only under its own unit tests.

* Log crosslink failures during AfterDephantomize instead of swallowing them

A crosslink error while filling a phantom blob (or one of its orphan/delta dependents) previously vanished silently, leaving storeReceivedFile report success with the artifact filled but un-crosslinked. Surface it via slog.Warn so real corruption on the fill path is at least visible, matching the non-blocking nit from review. check-in: 94919ca357 user: [email protected] tags: trunk

* fix: select xfer framing by Content-Type (#106); bound clone rounds by the client's decode limit (#109)

seeing which parsed, which made a decompression failure indistinguishable from "wrong framing" (the root of #104). Select framing from the §4 Content-Type instead: application/x-fossil is the §4.1 compressed container, every other type (including application/x-fossil-uncompressed, used by clone-v3 replies) is plain card text — the same rule canonical fossil applies. Verified against fossil 2.28: pull replies arrive as the prefixed container, clone replies as plain text, and no reply is ever an unprefixed raw zlib stream. The former unprefixed-zlib framing, checked first on the unverified belief that fossil emits it, was removed along with the length-prefix/zlib-header aliasing guard that only existed to disambiguate the two compressed framings trial-dispatch conflated. A compressed body that fails to inflate is now reported as the fault it is, never re-fed to the card parser. xfer.Decode takes the Content-Type; byte-oriented callers with no header pass the compressed type explicitly.

crossing one whole, so a round carried budget + one whole artifact. When sub-budget filler preceded a large artifact, that sum exceeded what the client could decode (xfer.MaxDecompressedBytes) and the clone failed with a spurious "declared container length exceeds" error whose threshold depended on incidental filler size. The server now flushes the round and sends an over-bound artifact alone when adding it would cross the client's decode bound, so clonability is a property of the artifact alone. The 2*budget compile-time guard now certifies what it appears to.

* fix: reject unrecognized xfer Content-Type with 415 (#106)

A POST whose Content-Type was absent or unrecognized fell through to the plaintext card decoder, which is exactly the #104 failure shape: possibly- compressed binary fed to the card parser. Request bodies come from untrusted peers, so a body carrying no framing signal is rejected rather than guessed at. Real fossil always sets the header; no conforming peer is affected.

xfer.KnownContentType shares the media-type parsing with the framing check, so the server and Decode agree on what a type means.

Also runs the #109 filler subtests in parallel — serially they pushed internal/sync past the 120s per-package timeout CI uses. check-in: f1236bd031 user: [email protected] tags: trunk

Issue #60's triage was stale: the live truncation bug it describes was already closed on main by 11967ff (#62), which added millisecond rounding to timeToJulian. What remained was a coverage gap, not a defect.

An earlier attempt on this branch also normalized the float64 branch onto the millisecond grid, on the theory that a driver-independent helper is strictly better. That was wrong, and this commit does not do it. A scanned float64 IS the value stored in the row, so snapping it can only move it away from what the row holds. Timeline's pagination cursor depends on the opposite property: its predicate `e.mtime < ? OR (e.mtime = ? AND b.rid < ?)` requires the cursor to compare exactly equal to the row it came from. SQLite's own julianday() computes iJD/86400000.0, a different expression from TimeToJulian's julianEpoch + ms/86400000.0, and the two land one ulp apart on a real fraction of instants — so normalizing alters mtimes written by upstream Fossil, the library's primary real-world input. Raised, the cursor row repeats as the next page's first row; lowered, rows are skipped. Both are silent wrong answers.

The float64 branch therefore stays verbatim, and its doc comment now records why that exactness is load-bearing rather than incidental. The same reasoning applies to tag propagation's `tagxref.mtime < ?`, where SQLite compares the bound value against raw stored floats: an exact bound is the correct one there too, so no normalized variant is introduced for it either.

The one genuine transform stays where it belongs, in the time.Time branch, which must round because ncruces hands back DATETIME columns carrying its own sub-millisecond conversion noise. Its comment now also records the limit of that recovery: an mtime written at finer than millisecond resolution cannot be reproduced through a time.Time at all, and rounding the float64 branch to match would discard precision on both drivers rather than recover any.

Non-functional cleanup: millisPerDay and julianDayFromMillis replace two open-coded copies of the same expression.

Coverage added (no build tag; these run under both drivers): - db: TestScanJulianDayFloatBranchIsBitExact sweeps julianday()-computed mtimes and asserts each scans back bit-for-bit, with a guard that fails the test if the sweep ever stops containing divergent values. Verified to fail against the normalizing version and pass against this one. - db: TestScanJulianDayIsDriverIndependent proves the float64 and time.Time representations of one millisecond-aligned write converge, with noise in both directions. TestScanJulianDaySentinelZero pins the "no mtime" sentinel that bisect's `mtime != 0` guard depends on. - tag: TestPropagateAllOrderingAdjacentMillisecond drives the `tagxref.mtime < ?` decision through ScanJulianDay with adjacent-millisecond mtimes; it fails under a truncating conversion and passes under the rounding one, and passes under modernc regardless — the reason this class went unnoticed twice.

Consumer classification (ScanJulianDay): - manifest/timeline.go NewCursor: ordering/equality, requires exactness - tag/propagate.go PropagateAll -> `tagxref.mtime < ?`: ordering (now covered) - tag/propagate.go childMtime -> mtimeQueue.Less: ordering, traversal-order only (visited set makes it result-neutral) - manifest/log.go, bisect/bisect.go: display-only

Known limitation, left alone deliberately: julianToTime truncates where timeToJulian rounds, so the two are not inverses at sub-millisecond resolution. Pre-existing, now documented; ScanTime's consumers are display paths with no ordering boundary. This commit does not widen the gap. check-in: 78a793dd83 user: [email protected] tags: trunk

* fix: return error when clone stalls with phantoms outstanding

Clone's convergence loop broke out of its round loop as soon as the phantom count stopped decreasing and returned success, even when the phantom table was still non-empty. That leaves the caller holding a repository missing content with no error and exit code 0.

Stopping on a stall is still correct — spinning forever would be worse — but reporting it as success is not. checkStop now returns a PhantomStallError when the loop halts with phantoms remaining, giving the caller a programmatically-reachable count plus a sample of the missing UUIDs for diagnosis. A clean clone still returns nil with an empty phantom table.

Fixes #64.

* fix: preserve partial CloneResult when Clone returns an error

Clone's named result return was never assigned on the error paths — cs.run's cloneResult stayed in a local variable, so a caller catching a PhantomStallError (or a crosslink failure) got a nil result even though a partial one, with the round/blob counts collected so far, was sitting right there. Assign it before returning on both paths.

* fix: make phantom stall error sample deterministic

newPhantomStallError previously took an arbitrary 10 entries from map iteration order before sorting, so the sample was nondeterministic across runs when more than 10 phantoms were outstanding. Sort all UUIDs first, then take the lexicographically-smallest 10, matching what the doc comment already implied.

Addresses a non-blocking nit from review on #64. check-in: e5be6a9199 user: [email protected] tags: trunk

* fix: zero-length file commit panic and stale missing-file carry-forward

Two related bugs in the checkin path:

#68 - Committing a zero-length file panicked in blob.Store, which treated an empty artifact (a normal, well-known Fossil blob) as invalid input. Worse, manifest.Checkin's postcondition defer fired unconditionally during the panic's stack unwind and re-panicked with its own unrelated "manifestRid must be positive" message, masking the real cause. Fixed both: blob.Store now accepts zero-length (and nil) content, and Checkin's defer re-panics with the original panic value instead of asserting a postcondition that was never going to hold mid-unwind. Fixing the panic surfaced a second bug in the same family: buildCommitFiles used len(Content) > 0 to detect "already loaded from disk," which can't distinguish a genuinely empty file from one not yet loaded, so a newly added empty file failed with "no RID for unchanged file." Replaced the length sentinel with an explicit loaded-set.

#79 - A tracked file deleted from disk without Unmanage/Remove was silently carried forward into the new manifest with its last-committed content, rather than being detected. Verified against fossil 2.28: canonical fails a commit when a missing tracked file is in scope (either the implicit "commit everything" case, or explicitly named), but leaves an out-of-scope missing file alone. buildCommitFiles now stats every in-scope tracked file before treating it as unchanged and errors clearly, naming the file, when it's gone; a file outside the commit's scope (relevant only to an explicit Enqueue) is left untouched exactly as before. Updated the two existing tests that had documented the old silent-carry-forward behavior as deliberately deferred to this issue.

* fix: add CHANGELOG entry and unwrap-safe stat checks for #68/#79

Adds the CHANGELOG entries the #79 acceptance criteria required for a user-visible behavior change (missing tracked file now hard-fails a commit instead of silently carrying stale content forward) and for the #68 zero-length-file panic fix.

Also switches checkin.go's two os.IsNotExist(err) checks to errors.Is(err, os.ErrNotExist), which correctly unwraps the %w-wrapped os.ErrNotExist that MemStorage.Stat returns. Production was already correct (OSStorage.Stat returns a raw *PathError), but the tests were only exercising a generic fallback error branch instead of the intended "tracked file missing from disk" message. check-in: 0351167ae5 user: [email protected] tags: trunk

* fix: clone emits cfile with stored content per sec7.2, not expanded file

Our clone server was building xfer.FileCard from cache.Expand(...) -- every artifact fully decompressed, and every delta materialized into full content -- where sec7.2 requires cfile cards carrying the repository's stored compressed content retransmitted verbatim. On real repositories this ran 80-220x more bytes on the wire than the protocol intends, and was the real blocker behind #88's clone-batching fix being unable to bound repositories much past libfossil's own size.

emitCloneBatch now reads blob.Load's stored bytes directly (full content for a whole artifact, delta bytes against DeltaSource for a deltified one) and emits xfer.CFileCard instead of xfer.FileCard, with DeltaSrc and USize (the full reconstructed size, from blob.size) populated correctly for delta rows. content.DeltaSource is exported so this and other callers outside internal/content can ask whether a rid is stored as a delta without duplicating the query.

No client changes were needed: session.go already routes CFileCard through the same handleFileCard path as FileCard, and storeDeltaContent already stores a received delta exactly as given via a create-phantom-if-missing lookup on its source, resolving delta-before-base arrival the way #53's phantom storage model was built for.

DefaultCloneBatchBytes is left at its current value rather than retuned toward canonical's 5,000,000 -- that constant's tuning is #109's territory now that bytesSent tracks stored bytes instead of expanded ones, and retuning it here risks colliding with that work.

* fix: expand deltified rows during clone instead of sending delta cfile

Code review reproduced a real interop regression: emitCloneBatch sends cards in ascending-rid order, but content.Deltify always deltifies the older artifact against the newer one, so a delta's DeltaSource almost always names a rid greater than the delta card's own rid. That means a delta cfile routinely reaches the client before the very source card it names. libfossil's own client tolerates the forward reference (storeDeltaContent phantoms the unseen source and resolves it later), but real fossil 2.28 does not -- the source stays a genuine, unfilled phantom, and content_get() on it during the client's post-clone rebuild pass returns without initializing its output blob, tripping an assertion in blob_copy (blob.c:397) and aborting the client mid-rebuild. Verified independently against a real fossil 2.28 binary cloning a libfossil server holding a delta chain.

emitCloneBatch now expands a deltified row to full content before sending it as a plain (non-delta) cfile, rather than forwarding the stored delta bytes with DeltaSrc set. The card is still zlib-compressed, so the wire-size win over the old FileCard/Expand path survives for every row; only the additional delta-retransmission savings are given up, and only for deltified rows. Re-enabling that needs either a source-before-dependent send order or confirmed forward-reference tolerance in canonical, and is left as a follow-up rather than blocking this fix.

TestEmitCloneBatchExpandsDeltifiedRows (renamed from TestEmitCloneBatchRetransmitsDeltaAsDelta) now pins the new contract: no DeltaSrc, full-sized payload. TestCloneRealFossilWithDeltaChain adds an end-to-end check against a real fossil binary, when one is on PATH. check-in: 0ca5133c62 user: [email protected] tags: trunk

* fix: bound delta.Apply growth per-command and cut ~5x allocation overhead (#81, #82)

Two related fixes to the same command loop in delta.Apply:

- #82: check len(output) against the declared target length after every copy/insert command, not just at the ';' terminator. A malformed delta with many copy commands could previously grow output well past both targetLen and maxDeltaTargetSize before anything noticed, since the only comparison happened at the end.

- #81: once a command has actually executed (proving the delta contains real work, not just an inflated header claim), upgrade output's capacity in a single step sized off len(delta) rather than relying on append's own ~1.25x large-slice growth factor to climb from the 64 KiB initial cap to the final size. That climb was landing near 5x the target size in cumulative allocation; the single upgrade restores near-exact sizing for honest deltas while keeping the tiny-payload/ huge-header attack closed, since a delta with no real commands never reaches the upgrade.

Added TestApply_LoopBoundsAgainstTargetDuringCommands (regression for #82: many copy commands exceeding a small declared target are rejected early, with allocation growth bounded near the declared target rather than the full command sequence) and TestApply_LargeTargetAllocationBounded plus BenchmarkApply_LargeTarget (regression/measurement for #81: a 1 MiB target now allocates ~1.06x its size instead of the prior ~5x).

Cross-checked against fossil 2.28 for wire-format compatibility; full non-dst suite and dst simulation suite (90 scenarios) pass.

* fix: address delta.Apply review nits from #81/#82

Rebuild the #81 regression test/benchmark around a many-small-copy delta shape (4096 copy commands into a 1 MiB target) instead of a Create()-derived single-big-insert delta. The single-insert shape allocates ~1.06x on both fixed and unfixed code (grows once), so it neither exercised the ~5x append-growth-climb regression nor would have caught a future reintroduction of it. Verified against a reverted delta.go: unfixed code now measures ~5.29x on this benchmark, fixed code ~1.06x.

Also move the len(output) > targetLen bound checks ahead of their append calls in both the copy and insert command branches, so an over-target command is rejected before its bytes are appended rather than after -- matches the #82 issue wording exactly, closing the (non-exploitable, since a single command's overshoot is already bounded and resident) gap the reviewer flagged as a nit. check-in: 355e7d9910 user: [email protected] tags: trunk

* manifest: create forumpost on demand, matching canonical rebuild

Crosslink's candidate query names the forumpost table unconditionally, but canonical fossil creates it lazily -- only once a forum artifact needs it -- and fossil rebuild drops it for a repository that never populated it. Crosslink now recreates the table itself before the sweep runs, matching canonical's on-demand behavior instead of erroring with "no such table: forumpost" against a rebuilt repo.

Audited the rest of the on-demand-in-canonical set (ticket, ticketchng) against the current schema: both are created eagerly by canonical at init and survive rebuild, so they don't share this problem.

Added TestCrosslinkForumPostTableDropped (drops the table directly) and TestCrosslinkAfterFossilRebuild (drives an actual fossil rebuild first) to cover the regression, and updated the existing fossilbin comparison test's comment now that the rebuild-last ordering is no longer required.

* manifest: visit crosslink candidates in delta-chain order

Crosslink swept candidates in ascending rid order, but content_deltify stores a blob's older versions as deltas against its newer ones, so delta chains run the opposite direction: high-rid root down to low-rid ancestors. An ascending sweep's first candidate of a chain therefore materializes the whole chain, and the rest are consumed arbitrarily far in the future -- the working set of a whole-repository sweep was the entire repository expanded (8.2 GiB for the Fossil SCM repository).

Candidates are now topologically sorted so a delta's base is always visited before its dependents (deltaChainOrder), matching Fossil's own rebuild_step shape: root first, each descendant one delta application from an already-expanded predecessor. The working set this sweep needs is now bounded by how many chains are in flight at once, not by repository size.

That reordering also changes the order artifact side effects get written in, since content expansion and artifact crosslinking share one loop. Most of what Crosslink writes turned out to already be a pure function of its own artifact (event/plink/mlink/tagxref-origin rows) and needed no change. Two things generalize across the whole plink graph rather than any one artifact and were genuinely order-sensitive:

- leaf table bookkeeping, which inserted the new checkin and deleted its parent -- correct only when parents are linked before their children, which delta-chain order no longer guarantees. - tag propagation, which ran once per checkin from its immediate parent -- reachable only to children plink already held at that moment.

Both are now full, order-independent repair passes (repairLeafTable, repairTagPropagation) run once after the sweep, mirroring Fossil's own manifest_crosslink_end + leaf_rebuild() shape: defer the order-sensitive work and repair it once rather than trying to preserve order through the sweep. tag.propagate's mtime-gated, plink-live-at-call-time design turned out to already be confluent -- replaying every self-declared tag origin once, in any order, converges on the same tagxref state -- so the repair is just doing that instead of depending on visiting order to get there implicitly. Tech-note event replacement and forum thread titles turned out to already have the same confluent shape and needed no change; left comments explaining why.

Also fixes a tag.PropagateAll crash on any origin with a NULL tagxref value (e.g. CrosslinkCluster's cluster tag), surfaced by the new propagation repair actually reaching those rows for the first time.

Regression coverage: a real three-generation checkin chain, forced into a genuine manifest delta chain, that would have left two of three checkins untagged and all three marked leaf under the old per-checkin propagation and incremental leaf bookkeeping.

* fix: address review nits on crosslink order-sensitivity and delta cycles

- deltaChainOrder now returns an error instead of panicking when the candidate delta graph fails to drain. The delta table is on-disk data that can arrive over sync from an untrusted peer, so a cycle (corrupt or hostile repo) should surface as an error rather than crash the process.

- snapshotDerived's digest now covers tagxref (rid/tagid/tagtype/ srcid/origid/value), and TestFossilBinaryReadsCrosslinkedRepo compares it against canonical fossil rebuild -- closing the gap the test previously called out by name ("tagxref is the notable gap").

- Added TestFossilBinaryReadsCrosslinkedRepoBranchAndMerge, exercising the order-sensitive paths incrementalHistory's single-branch history cannot reach: a feature branch diverging from trunk, and a merge commit whose own branch tag must come from its primary parent, not the merged-in one. event/plink/tagxref match canonical fossil row-for-row on this topology.

leaf and mlink are deliberately excluded from that comparison: the fixture surfaced that both have pre-existing gaps against canonical fossil (repairLeafTable counts any plink edge rather than only primary-parent edges; insertCheckinMlinks writes a row for every F-card rather than only changed ones) that predate this fix and are out of scope for #102/#103.

* fix: store received wire frames verbatim instead of re-encoding

Clone reproduced content correctly but not bytes: the receive path decompressed every cfile frame and re-compressed it with blob.Compress's unparameterized zlib writer, so a clone's blob.content never matched its source's — about 0.5% larger on the wapp corpus, purely from re-encoding frames that were already correctly encoded on the wire.

xfer's cfile decoder now hands back StoredBlob alongside the decompressed Content: the same bytes recast into Fossil's on-disk blob format ([4-byte BE size][zlib data]) without a second zlib pass — verbatim when the wire already sent that format (Fossil's send_compressed_file case), or the received zlib stream with a length prefix bolted on when it didn't (our own encoder's plain-zlib case). storeReceivedFile/storeDeltaContent/ storeResolvedContent and blob.StoreDeltaRaw thread that through via a new blob.EncodeForStorage helper, which writes the verbatim bytes when given and only falls back to Compress when there's nothing to preserve — covering both the clone/pull and server-push receive paths.

Locally authored content (never received over the wire) always takes the Compress fallback, so #71's deltification policy is untouched. The clone BUGGIFY corruption test drops storedBlob alongside the corrupted content it's exercising, so verbatim storage can't mask the corruption it exists to catch.

Adds a corpus-level regression test cloning a real multi-revision fossil repo and asserting sum(length(content)) matches between source and clone.

* test: use RequireFossilBin instead of removed HasFossil helper

testutil.HasFossil was removed upstream in favor of RequireFossilBin; this test's skip guard needed the same update to compile against current main. check-in: cc42ad8829 user: [email protected] tags: trunk

* test: route fossil-binary tests through one REQUIRE_FOSSIL_BIN helper

REQUIRE_FOSSIL_BIN=1 turns a missing fossil binary into a test failure instead of an invisible skip, but only internal/manifest honored it. The six other fossil-dependent sites each did their own exec.LookPath + unconditional t.Skip, so on those a run with no fossil was byte-identical to a passing one.

Add testutil.RequireFossilBin, which resolves the binary the same way FossilBinary does (FOSSIL_BIN override, then PATH) and either skips or fails based on REQUIRE_FOSSIL_BIN. Route testutil.NewTestRepo, both internal/delta CLI round-trips, the four internal/merge validation tests, the internal/xfer sanity check, and the ls-branch-tip repro through it, deleting their bespoke lookup+skip logic. CI already sets REQUIRE_FOSSIL_BIN=1 on the unit-test steps, so this makes all seven sites fail loud on absence while local runs without fossil stay opt-in.

* test: route every fossil-binary gate through RequireFossilBin

The first pass left six sites still deciding for themselves whether a missing fossil binary is a skip or a failure. Three of them (internal/sync's startFossilServer and its two integration tests, manifest's two rebuild tests, the root TestPhaseB_Integration, and hash's TestSHA1_FossilValidation) skipped unconditionally, so with REQUIRE_FOSSIL_BIN=1 and no fossil on PATH those packages still reported ok -- the exact silent pass the flag exists to prevent. The other three had hand-rolled copies of the flag check, which is the duplication that let the original drift happen.

All of them now call testutil.RequireFossilBin, and HasFossil is gone: it was a second, skip-only way to ask the same question, and it is what produced the sites that ignored the flag.

Also validate the FOSSIL_BIN override with LookPath, so a stale path reports "fossil not found" instead of surfacing later as a raw fork/exec error.

* test: resolve fossil binary via RequireFossilBin at remaining call sites

* fix: correct stale module path missed by rebase auto-merge

Three imports added by earlier commits in this branch predated the rebase target and had no corresponding line on main to conflict against, so git's auto-merge carried the old module path through verbatim. check-in: f804878bd3 user: [email protected] tags: trunk

* fix: degrade unparseable clone VERSION/SEQNO instead of erroring (#95)

parseClone routed both tokens through strconv.Atoi before the §8.1 digit-only rule ever got a chance to apply, so a token Atoi could not parse at all (not just one that fails digit-only) hard-errored the whole message instead of degrading gracefully. Carry the raw tokens forward as an UnknownCard on parse failure, matching how UnknownCard already handles a non-decimal clone_seqno, so a later card in the same message still decodes.

* fix: reject an oversized §4.1 declared length before trying raw zlib (#110)

xfer.Decode tried the unprefixed-zlib framing before ever reading the §4.1 container's declared length prefix. A peer declaring a length at or above 0x08000000 produces a prefix whose first two bytes present as a structurally valid zlib CMF/FLG header (RFC 1950), so a large declared length aliases as the start of a raw zlib stream and gets first refusal on a legitimate compressed container. Read the declared length whenever the body opens with a valid zlib header and reject it against MaxDecompressedBytes before attempting format 1 at all. check-in: 2b9694f525 user: [email protected] tags: trunk

The internal/hash import was added by the #93 T-card validation fix on a branch cut before the go-libfossil module rename merged. The rename's automated rewrite only touched import lines that existed at the time, so this newly-added import kept the old path and broke go build ./... on main after both merged.

CI's go build ./cmd/libfossil/ and go test $(go list ./... | grep -v /dst) didn't catch it: go list ./... drops a package that fails to compile from its output rather than failing the command, so the broken package silently dropped out of the test run instead of failing the build. check-in: 7764ecf6ff user: [email protected] tags: trunk

* fix: J card decodes value, stores name verbatim (#90)

The J-card parser had the escape direction inverted: it decoded the field name and exposed the value with its escape sequences intact. Both the spec (4.7.8) and canonical fossil (manifest.c) do the opposite -- defossilize only the value, compare the raw name.

Swap both the parse and marshal directions so the value round-trips decoded and the name stays verbatim.

* fix: T card escape-encodes the tag name on marshal (#91)

A tag/branch name containing a space was written verbatim, so 'my branch' produced 'T *sym-my branch *' -- a structurally valid but silently wrong card that re-parses as two tokens.

Align with canonical fossil: escape-encode the name on marshal and escape-decode it on parse, leaving the target token verbatim in both directions (4.7.16). TagCard.Name now holds the decoded form, so tagOrderKey reattaches the sign without re-decoding.

* fix: enforce two skipped T-card §4.7.16 checks on parse (#93)

parseTCard validated the sign and ordering key but accepted two malformed shapes from untrusted peers:

- a name that after the sign is entirely hex digits, indistinguishable from a hash target; - a target that is neither a valid artifact hash nor the literal '*'.

Reject both with errors (never a panic). hash.IsValidHash reuses the same target check the primary uuid path uses; the '*' and non-empty guards run first since it panics on an empty string.

* fix: T card decodes and encodes the tag value (#91), guard hex names on marshal (#93)

§4.7.16 applies the same escape rule to all three T-card tokens: the name and value are escape-decoded on parse and escape-encoded on marshal, while the target is stored verbatim. Only the name was handled; the value was passed through raw in both directions.

The read side was the real damage. `fossil branch new 'my branch'` writes

T *branch * my\sbranch T *sym-my\sbranch *

so importing an ordinary fossil repo with a spaced branch name produced a branch literally named "my\sbranch" through crosslink. Verified against the fossil binary: the artifact for that command now parses to value "my branch" and re-marshals byte-for-byte identical.

Marshal also gains the mirror of the hex-only tag name check parse already enforces. Without it a caller could build and store an artifact this repository could never read back; canonical fossil never emits one either, since it prefixes branch and user tags with "sym-". check-in: 0e579e24a4 user: [email protected] tags: trunk

* fix: correct go.mod module path to go-libfossil (#76)

The module directive still declared github.com/danmestas/libfossil after the repository was renamed to go-libfossil. pkg.go.dev/godoc scraping matches the declared module path against its hosting path, so the stale path broke SDK doc generation.

Rename the module path across all four in-repo modules (root, the two db/driver submodules, observer/otel, and docs/site) and every internal import, plus the release workflow, the doc-generation script, and the docs.

No tag has been published under the go-libfossil path yet, so the cross-module requires cannot resolve from the proxy. Add replace directives pointing the in-repo requires at their local directories so GOWORK=off builds and the CI matrix resolve without a published version. These replaces are ignored by downstream consumers and can be dropped once new-path tags are cut.

* Guard releases against replace directives; regenerate SDK docs

The go-libfossil rename bridges the in-repo modules with `replace` directives. Go refuses a main module that carries them, so a tag cut while they are present breaks `go install .../cmd/libfossil@<tag>` and every `@version` fetch of the driver submodules. Add a Release workflow step that fails the job before anything is published if any tracked go.mod still contains a replace directive, so the bridge is a CI-enforced TODO rather than a silent landmine. The step comment records what unblocks a release: re-tag the submodules with the corrected module paths, bump the root requires, then drop the replace blocks.

Regenerate the SDK reference docs. The previous pass only substituted URLs over the pre-existing generated files, leaving stale signatures (Repo.Timeline was still documented as taking LogOpts rather than TimelineOpts). Note that gomarkdoc must be run outside a git worktree — inside one it silently strips every source link.

Also reword the README rename note, which read as though the import path had not changed, and re-sort the ncruces require block. check-in: 6273a0750d user: [email protected] tags: trunk

deltifyBreaksLoop bounded its ancestor walk on deltifyMaxChainWalk (1,048,576), 64x the maxDeltaChainDepth (16,384) cap the read-path walks use, with no rationale for the difference. It is the same delta table over the same chains, so the smaller bound's reasoning applies verbatim.

Bound deltifyBreaksLoop on maxDeltaChainDepth and remove the now-unused deltifyMaxChainWalk constant. Behaviour is unchanged for any real corpus -- the bound only bites far beyond the deepest observed chain (2,546). check-in: 17724bfdfd user: [email protected] tags: trunk

* fix: never parse a compressed body as cards (#104)

A clone served by libfossil failed in libfossil's own client with

xfer: decode card 16303: xfer: empty line after split

on a response that contained 202 cards. The index, the card, and the fault were all fictional.

xfer.Decode tried three body framings in order -- raw zlib, the §4.1 4-byte-length container, then uncompressed card text -- and treated any decompression failure as "this is not that framing". decompressBounded failed a well-formed container purely because its decompressed length passed a 50 MB bound, so the fallback chain handed the still-compressed bytes to the card parser. The parser read compressed noise as cards for sixteen thousand iterations before one of them split into no fields, and reported that as the fault. Captured at offset 15,084,752 of the failing body, the "card" the parser was reading is DEFLATE output:

"...\xd9\x34\x50\xe7\xcf\xa0\xf8\xd5\xd5\x87l\xff\t\xde\xfb\xe3..."

Two changes. Once a body is recognized as a zlib stream, a failure to decode it is reported rather than reinterpreted -- oversize, truncated, or corrupt, the error now names the transport failure instead of an arbitrary card position. Both compressed framings are still attempted, since a length prefix can coincidentally parse as a zlib header, and uncompressed bodies still reach the card parser; only compressed ones no longer can.

The bound itself was also wrong, and is the reason this was reachable at all. §4.1 places no limit on the decompressed size of the container; the 50 MB was a local memory guard that sat below what this implementation's own server emits in one clone round, so two libfossil peers could not clone from each other even though each could clone from fossil. It is now 64 MiB and coupled to sync.DefaultCloneBatchBytes by a compile-time guard that fails the build, rather than the clone, if the budget is raised past half of it.

The defect predates the size-bounded clone batching of #100. #100 shrank rounds enough to hide it on the corpus that exposed it; it did not touch the decoder.

Regression coverage, in two layers. internal/xfer pins the decode behaviour directly: an oversize container, a truncated one, a large in-bounds one, and an uncompressed body. internal/sync adds the configuration no existing test covered -- libfossil serving and libfossil cloning across a real HTTP encode/decode, on a corpus of multi-megabyte incompressible artifacts that crosses the server's per-round budget. Every prior clone test was either served by fossil or wired through MockTransport, which passes *xfer.Message values across untouched and never encodes a wire body at all. Artifact size is the other half of the reproduction: this failed on wapp, whose artifacts are large, and not on pikchr, whose artifacts are small.

Verified against the wapp corpus: the round trip that failed now completes with 1038 of 1038 artifacts on both ends.

* fix: make the #104 fixture reach the bound, and pin the bound from both sides

Review found the flagship regression test passed against pre-fix code. It did. Its corpus was 21.6 MB spread over twelve artifacts, and pagination keeps every round near the 16 MB budget however large such a corpus grows, so the round never came near the 50 MiB bound it was supposed to cross. The 25-line rationale reasoned about crossing DefaultCloneBatchBytes, which is not the quantity that matters and never was; a permanent comment asserting a mechanism that does not exist is worse than no comment.

The quantity that matters is one artifact. A round is charged against the budget before each artifact and sends whole the artifact that crosses it, so a round carries budget plus one entire artifact of unbounded size. A single artifact larger than bound-minus-budget is the only corpus that can exceed the bound, and it is also the one path by which a round can still exceed the current bound -- so the fixture guards the live risk and reproduces the historical one with the same bytes. It is now one incompressible artifact of 7/8ths of MaxDecompressedBytes, sized against the constant so it tracks it, behind testing.Short.

Verified against a pre-fix tree with both test files dropped in:

TestCloneSelfRoundTripOversizeArtifact FAIL Clone: handleFile aad935c2...: UUID mismatch for received file: expected aad935c2..., got 501b4814...

which is the sharper end of this defect: pre-fix did not merely misreport, it accepted the compressed bytes as cards and produced an artifact whose content did not hash to its name.

The cheap end-to-end round trip is kept as a separate test and described honestly as a coverage plank -- the first test anywhere here to have libfossil decode a body libfossil produced -- rather than as a guard on anything.

Second finding, message.go: the claim that format 1 is the "Fossil HTTP sync wire format" is false. Every application/x-fossil body carries the §4.1 length prefix, request side and reply side, both through blob_compress. Format 1 matches nothing canonical emits and is kept as a tolerance, now labelled as one.

Its safety was also load-bearing on a constant with no stated relationship to it. For a container of decompressed length N, prefix byte 0 is N>>24, and a zlib CMF requires CM==8 in the low nibble, so no container under 128 MiB can present a valid header at offset 0. Raising MaxDecompressedBytes past that point would leave decompressContainer's prefer-what-decodes ordering as the only remaining defense -- a narrower margin than this code was written against, and #98 is exactly the change that might want a different bound. That is now a build failure rather than a trap, matching the guard already on the other side. Verified to fire at 0x08000000 and to permit 0x07FFFFFF.

Also: the MaxDecompressedBytes doc claimed to stop a hostile body from exhausting an embedding process's memory. It does not -- the transport reads the body whole before Decode sees it. It bounds expansion only, and now says so.

Test hygiene: the server handshake polls for a live listener instead of sleeping 200ms and dropping ServeHTTP's error, and the 32 MiB and 64 MiB xfer allocations skip under -short like their sync counterpart.

* fix: correct three claims about what the #104 guards cover

The fix is unchanged. Three statements about it were wrong, and a comment that misstates a mechanism is worse than no comment because the next reader trusts it.

The ordering claim was the serious one. decompressContainer was described as "preferring whichever framing actually decodes, so a spurious header match still loses to a real container". It does not. Format 1 returns immediately on success and format 2 is reached only when format 1 fails; the error comparison at the end selects which failure to report when both have already failed, and protects nothing when format 1 spuriously succeeds. Probed: a bare zlib body returns from format 1 with format 2 never attempted.

That makes the aliasing guard weaker than its comment claimed, and in a direction neither review had reached: the aliasing byte is the top byte of the length the PEER declared, and this decoder never reads that field -- it slices past it and counts bytes actually inflated. No value of MaxDecompressedBytes constrains what a peer declares. A container declaring 0x081D0000 while carrying 46 bytes presents a parsing zlib header at offset 0, measured.

What actually saves a real container in that case is the stream, not arithmetic: after two prefix bytes are eaten as a header, inflate must still parse the remaining two plus the real stream as DEFLATE and match its Adler-32. Measured `flate: corrupt input` at all three aliasing lengths probed, so format 1 fails, format 2 runs, and the container decodes correctly.

The guard is kept because it states something true and useful -- Encode writes uint32(len(raw)) as the prefix, so holding the bound below the threshold means no body libfossil emits can carry an aliasing prefix -- but it is now documented as an invariant on our output, not a defence of our input, with the airtight alternatives named and left to their own issues.

Second, "a single artifact larger than this bound cannot be cloned" is true but not the operative threshold. A round already holding bytes has that much less room, so the ceiling is the bound minus what the round accumulated first. Measured on this branch with a 58,720,256-byte artifact: 8, 12 and 15 MB of filler ahead of it all fail, and 17 MB passes because it exhausts the budget and pushes the artifact into a round of its own. Non-monotonic in the filler, which is worth knowing. That is #109, a defect this fix exposes rather than causes; the fixture's claim to guard it is withdrawn and it now says plainly that it covers the leading-artifact case only.

Third, the changelog's "pin the bound from both sides" overclaimed. The guards pin the bound against the batch budget and against the aliasing threshold. Neither certifies a round.

Rebased onto 8b6da9a. The internal/content -race failure reported last round was not a flake and not unrelated -- this branch predated #107, which rewrote available.go. TestIsAvailable_CyclicChainTerminates goes from exceeding its 30s budget to 0.06s across that rebase.

---------

Co-authored-by: Dan Mestas <[email protected]> check-in: c892c7a622 user: [email protected] tags: trunk

* perf: memoize delta-chain interiors during crosslink (#89)

manifest.Crosslink ran at roughly 1.4 events/sec on the Fossil SCM repository, which puts a 27,000-artifact clone at two and a half hours of CPU after the transfer has already finished. A CPU profile of a sweep over that corpus attributes 152.77s of Crosslink's 167.20s -- 91% -- to content.Expand, and 126.35s of that to the single blob.Load inside its delta-replay loop.

The cause is chain depth, not per-event overhead: not a transaction per event, not a missing index, not a full-table scan. content_deltify stores a file's older versions as deltas against its newer ones, so one file's history is a single chain -- 469 nodes deep on average across that repository, 1,032 for the blobs carrying artifacts, 2,546 at worst. Expand walked each chain from its root on every call and Crosslink calls it once per blob, so the sweep performed 31.7 million delta applications to expand 67,451 blobs.

Cache.Expand now stops the chain walk at the deepest ancestor it already holds and keeps every node it materializes on the way forward, and Crosslink runs one such cache for the sweep. Blob content is immutable once written and the candidate rids are snapshotted before the loop, so nothing a sweep does can invalidate an entry.

Every node handed to the cache is verified against the UUID its blob row is stored under before it is stored, because a cached interior is served later as if it had been expanded in its own right. The number of hash checks per sweep does not change: each blob is materialized once and verified once, where before each was expanded once and verified once. A plain Expand, with no cache, walks and verifies exactly as it did.

IsAvailable folds its two-statements-per-chain-node walk into one. It runs once per F-card of every check-in being crosslinked, over the same deep chains, and every SQL round trip there pays a WAL read-lock acquisition -- the round trips, not the work they do, are the cost.

No transaction boundary moves and no batching is introduced. A manifest that fails to crosslink still fails on its own, exactly where it did.

Measured on the Fossil SCM repository, 67,451 content blobs and 27,060 artifacts. Time for Crosslink to write its first 300 event rows: 221s before, 131s after. Over the expansion path in isolation, sweeping the same corpus in rid order: 22.9 blobs/sec with no cache, 50.1 at the 256 MiB budget set here, 414 at 1 GiB -- where the process needs 2.9 GB resident, which is why 1 GiB is not the number.

A bounded cache cannot close the gap on its own. Chains run from a high-rid root down to low-rid ancestors while the sweep visits rids ascending, so the first candidate of a chain materializes all of it and the remaining nodes are consumed arbitrarily far in the future; the working set of a whole-repository sweep in rid order is the whole repository expanded, 8 GiB for this corpus. Closing it means visiting candidates in delta-chain order, which is how Fossil's own rebuild avoids the cost, and which changes the order-sensitive state Crosslink writes -- leaf, tag propagation, tech-note event replacement, forum thread titles. That is a separate change.

TestFossilBinaryReadsCrosslinkedRepo covers the acceptance criterion this cannot self-certify: a repository whose relational tables were written through the accelerated path passes fossil 2.28's test-integrity, shows its tip in fossil timeline, survives fossil rebuild, and its event, plink, leaf and mlink rows match, column for column, what fossil derives from the same blobs.

* fix: bound and cycle-check the delta-chain walks (#89)

IsAvailable walked the delta chain with no cycle detection, to a bound of 100,000 against real chains whose deepest observed node count is 2,546. It is wire-reachable -- sync/clone.go, sync/handler.go, and once per F-card of every check-in Crosslink processes -- so a peer that plants a two-node delta cycle bought a 100,000-iteration query loop on every call, inside whatever process embeds this library.

Both halves are fixed. Each walk now carries a visited set and stops at the first repeat, which is the actual defence: a cycle costs its own length, not the bound. maxDeltaChainDepth replaces the old constant at 16,384, roughly six times the deepest chain measured on the Fossil SCM repository, and it is now a backstop for an acyclic-but-absurd chain rather than a number an attacker spends. walkDeltaChain, which the memoizing expansion path made hotter, previously had no bound at all and now shares the same one.

This is what made TestIsAvailable_CyclicChainTerminates sit at ~99% of its 30s budget on main and fail outright under -race here: the cyclic walk was running the full bound. It now finishes in about a second under -race, six runs out of six.

walkDeltaChain returns the content it stopped on instead of leaving expandChain to ask the store a second time. The truncation decision and the use of it were separate steps, so a concurrent eviction between them would send a delta row to blob.Load and hand raw delta bytes to the replay as full text. bindToName caught that and failed closed, but the caller saw a hash mismatch on an artifact that was not corrupt, and Crosslink's error path drops those silently. Unreachable while the cache is a single-goroutine local, but Cache is documented concurrency-safe and takes its mutex on every method, so the type promised something the expansion path did not deliver.

Cache entries are clipped to their length before being stored, so curSize is the bytes actually retained rather than an allocator-dependent approximation of them.

Test corrections: the corpus harness no longer closes the repository handle out from under the sweep goroutine when it returns at the event target; the sweep owns the handle and closes it. snapshotDerived now fails on an empty digest rather than comparing "" to "" and passing without examining a row. The fail-closed injector matches the merged chain-step statement specifically rather than any query mentioning delta, and says what it can and cannot prove now that the two lookups are one statement. The fossilbin comparison documents that it is not full-table equivalence, and which tables it leaves uncompared.

MEASUREMENTS, SUPERSEDING THOSE IN 2426591

The figures in the parent commit were taken before clipping, and its "221s -> 131s" pair predates the double-hash and IsAvailable fixes that landed in that same commit. Everything below was re-measured on the code as it ships, one run at a time on an idle machine, against the Fossil SCM repository (67,451 content blobs, 27,060 artifacts).

Crosslink to its first 300 event rows, cpuprofile on throughout:

original code 221.4s as shipped 103.9s 2.1x

Expanding the first 4,000 blobs, which isolates the expansion path:

budget throughput peak RSS entries held none 22.1 blobs/s 169 MB 0 0 256 MiB 52.0 blobs/s 1036 MB 543 255 MiB 1 GiB 468.3 blobs/s 2278 MB 15,842 1021 MiB

Clipping changed neither throughput nor memory. Crosslink to 300 events runs 103.9s clipped against 105.9s unclipped, peak RSS 1067 MB against 1050 MB, and the cache holds the identical 543 entries and 255 MiB at a 256 MiB budget either way -- at these sizes the buffers already arrive tightly sized. It is worth keeping as an accounting guarantee, not as a saving, and the comment on clip says so.

The earlier claim that capacity slack was why a 256 MiB cache reached nearly a gigabyte resident was wrong. Peak RSS tracks the no-cache baseline plus roughly twice the budget, because the cache is live data and the collector sizes the heap against it: 169 + 2x1021 = 2211 against 2278 observed at 1 GiB. That is why the budget stays at 256 MiB -- a gigabyte of cache costs over two gigabytes resident to clone a 72 MB repository. The shape of the curve did not move.

* docs: correct the claims around the chain bound and the RSS model (#89)

Three claims did not survive checking, and one bound was a node out.

maxDeltaChainDepth said it bounds "every delta-chain walk in this package". It does not: deltifyBreaksLoop is a third walk, under deltifyMaxChainWalk, which is 64 times larger. The rationale on the new constant applies to that walk just as well, but changing it would change deltify's shipped behaviour, so the sentence is narrowed to the two read-path walks it actually governs and points at the third rather than quietly covering it.

Two comments still named maxAvailabilityChainDepth, which no longer exists, and argued from IsAvailable having a bound "of its own" -- the framing of a world where the two walks had separate bounds. Against a shared constant that argument reads backwards. Both now name the shared bound, and the reason deltifyBreaksLoop keeps its own visited set and its own step cap is stated as the general rule it is: every walk over the peer-shaped delta graph carries both, and none of them relies on another walk's guard.

The retraction in the previous commit replaced one unsupported mechanism with another. "Peak RSS runs at roughly the baseline plus twice the budget" is a two-point fit whose worse point is the shipped one: it is 3% off at 1 GiB and 53% under at 256 MiB, where resident memory is four times the budget rather than twice. The measured table stays; the model is demoted to the direction the data supports, that peak RSS grows faster than the budget, with the fit's own weakness stated.

walkDeltaChain counted loop iterations where IsAvailable counts nodes visited, so the shared bound bit a node apart: a chain of exactly maxDeltaChainDepth+1 was unavailable to one walk and complete to the other. It now counts nodes, and TestChainBoundBitesAtSameLength pins the agreement at both boundaries -- nothing else ties the two loops together. check-in: 9864120a32 user: [email protected] tags: trunk

* fix: bound clone batches by output size, not artifact count (#88, #92)

The xfer server paginated a clone by a fixed 200-artifact count. A count cannot adapt to artifact size: 200 control artifacts and 200 large blobs produce wildly different messages, and neither corresponds to what the transport can carry. It capped a libfossil-served clone near 20,000 artifacts against our own MaxRounds=100 client, and left the response size unbounded in the other direction -- serving libfossil's own repository produced a single 44.9 MB round.

emitCloneBatch now accumulates the wire bytes it emits and stops once they reach DefaultCloneBatchBytes, mirroring canonical's `while( mxSend > blob_size(pOut) )`: the budget is tested before each artifact and the artifact that crosses it is still sent whole, which is what keeps the cursor advancing when one artifact exceeds the entire budget. The card header is charged alongside the payload so a repository of zero-length artifacts still paginates. Section 8.2 leaves the bound to the server ("clients MUST NOT infer count or size from range").

The budget is 16 MB rather than canonical's 5,000,000 max-download because this clone path emits fully expanded artifacts where canonical retransmits stored compressed form (Section 7.2) -- 80x to 220x the bytes on real repositories. Measured on libfossil's own repository (17,218 artifacts): 87 rounds and a 44.9 MB worst-case response before, 65 rounds and 24.7 MB after. Canonical's constant applied to expanded bytes would have needed 184 rounds, breaking clones that work today.

Adds the interop half: a canonical fossil 2.28 client cloning a generated corpus from `repo serve`, sized past the pagination boundary, checked for round-trip count, artifact parity with the source, and `fossil test-integrity`. The many-small-artifacts case is what discriminates the two bounds -- 2,056 artifacts clone in 3 round-trips under the size bound and 23 under the count bound. Corpus size is a parameter since the interesting behaviour is at the boundary. A missing fossil binary skips, or fails under REQUIRE_FOSSIL_BIN=1.

* review: state the ceiling honestly, fix the overhead comment

The size bound replaces an artifact-count ceiling with an expanded-bytes ceiling. It is a better ceiling and it beats the count bound on every corpus measured, but fossil- and sqlite-sized repositories still cannot be cloned from a libfossil server within MaxRounds. Record the full three-repository round table on DefaultCloneBatchBytes and say outright that only Section 7.2 removes the ceiling rather than moving it.

State the memory property as budget plus the largest artifact, not as a 24.7 MB ceiling. The crossing artifact is sent whole, so a repository holding one 2 GB artifact still emits a 2 GB round, and concurrent clones multiply it. 45 MB -> 25 MB is a large reduction of a pre-existing unbounded allocation, not a cap.

cloneCardOverheadBytes claimed to cover the hash and then len(uuid) was added on top, double-charging it. Drop the addend; 96 over-approximates the ~58-byte framing on its own and the slack absorbs the uncharged PrivateCard.

Also: smallBatch sets a one-byte budget rather than a batch size of one, so correct the comments that still describe the old knob; drop the post-loop multi-batch > single-batch check, which passes under both bounds and so discriminates nothing; and make serveRepo report a server that exited early instead of signalling into an uninstalled handler, where SIGINT's default action would kill the test binary outright.

* test: count clone batches on the server, not round-trips on the client

CI (fossil 2.23) and this machine (2.28) disagreed on an identical server: the multi-batch clone reported 1 round-trip against CI and 9 locally. The clone was correct in both cases.

fossil prints "Round-trips: N" carriage-return separated, once per round. 2.28 guards those prints behind bOutIsTty and adds a single final line for non-TTY output, so a captured clone yields exactly one occurrence holding the total. 2.23 has no such guard and emits every round into the capture, so the regex's first match was always 1 no matter how many rounds ran. Verified by cloning one repository from a real fossil server with both binaries: 2.28 prints "Round-trips:" once, 2.23 four times, first line 1 and last line 2.

So the assertion silently meant different things on different clients, and many-small-artifacts -- the only case that discriminates a size bound from a count bound -- was passing under 2.23 for the wrong reason, inert rather than green.

Count the clone batches the server emits instead. That is the property under test, it cannot be distorted by client-side reporting, and it is sharper: many-small-artifacts is 2 batches under the size bound against 22 under a reinstated count bound, where the round-trip numbers were 3 against 23.

This serves via sync.ServeHTTP directly so the handler can be wrapped. RepoServeCmd.Run is a five-line wrapper around the same call and TestRepoServeCmdCanonicalFossilClone still drives the command end to end. It also drops these subtests' use of the process-wide SIGINT. check-in: 37271d3f68 user: [email protected] tags: trunk

* Add libfossil version command

Sources the version string from runtime/debug.ReadBuildInfo (module version for go-install/tagged builds, falling back to a short VCS commit), with a -ldflags override for release builds where BuildInfo alone would collapse to "(devel)". Output is a fixed, single-line, four-field shape ("libfossil <version> <go version> <os>/<arch>") so a caller can rely on strings.Fields(Version()) unconditionally -- the motivating consumer stamps this into every emitted benchmark record.

No global --version flag: repo extract already owns --version as a command-scoped flag (source version to extract), and kong's global flags are visible in every subcommand's context, so a root-level --version would collide with it. version is a subcommand only.

Verified kong's existing parse-error path already exits non-zero (exitUsageError, 80) for unrecognized commands and flags against the pinned kong version -- added regression tests locking that in rather than changing behavior that was already correct.

* Reject malformed -ldflags version overrides at startup

buildVersion was used verbatim with no validation, so a whitespace- or newline-containing -X override silently broke Version()'s promised single-line, four-field contract: a space produced six fields instead of four, and a newline broke the single-line guarantee outright. The stated consumer is a benchmark harness parsing this line as a stable record, so a corrupted version field silently corrupts every record from that build in exactly the field meant to identify it.

A bad -X value is a build-time operator mistake, not a runtime condition, so it's rejected at package init rather than sanitized or worked around: sanitizing would silently mangle the operator's intent, and falling back to build info would report a version that is confidently wrong. Both would hide the mistake in the one field whose job is to be trustworthy. Failing the whole binary at startup surfaces a bad build the first time it runs, for any command, not just when someone happens to run `version`.

Also simplified versionFromBuildInfo: verified empirically (go1.26, `go build .` inside this checkout, with and without -buildvcs and inside/outside a VCS checkout) that Main.Version is only ever the uninformative "(devel)" in exactly the cases where vcs.revision is also absent from BuildInfo.Settings -- whenever a VCS revision is available, Go has already folded it into Main.Version as a pseudo-version. The separate vcs.revision fallback loop was dead code that could only ever duplicate a hash already present in Main.Version; removed it along with the now-unused shortCommit helper. check-in: 725c694cb9 user: [email protected] tags: trunk

* fix: remove StatusOpts, MergeOpts, CheckoutOpts — no call sites

No function in the package accepts any of these three types, and nothing constructs one anywhere: Checkout.Status() takes zero arguments, Repo.Merge takes positional string arguments, and CheckoutOpts had no construction site at all. A public type with no call site documents a capability that does not exist -- a consumer reading MergeOpts{Strategy: ...} in the docs could reasonably conclude a strategy-selecting merge API exists; it never did.

Repo.Merge and Checkout.Status keep their current signatures. An options-struct refactor for Merge was considered and declined.

CheckoutOpts.Force was also a third same-named Force field in this package, alongside the already-removed UpdateOpts.Force (#58) and the real, fully-wired ExtractOpts.Force -- removing it also removes that readability hazard. ExtractOpts.Force and its plumbing (checkout.go Extract -> internal/checkout.ExtractOpts -> extract.go's Force check) are untouched.

* docs: match em dash style in CHANGELOG breaking-change bullet

The new StatusOpts/MergeOpts/CheckoutOpts entry used ASCII double hyphens where the adjacent CloneOpts and UpdateOpts.Force bullets in the same file use a real em dash. check-in: 1a33f22119 user: [email protected] tags: trunk

* fix: clone_seqno is server-to-client only (#74)

The clone client transmitted a `clone_seqno` card to carry its pagination cursor. That card travels the other way: canonical's page_xfer() only ever writes it (src/xfer.c:1571) and has no parser for it, so a real fossil server answered `bad command: clone_seqno N` and the clone died on round two. Repositories that fit in a single batch never reached the pagination path, which is why wapp and pikchr cloned while fossil and sqlite did not.

The request cursor belongs on the clone card itself — `clone VERSION SEQNO`, read from token 2 at src/xfer.c:1553 — and the response cursor belongs on `clone_seqno`, which the client only consumes. Give each direction a single owner:

- The client sends its cursor on the clone card and never emits a clone_seqno card. It already consumed the server's card correctly; it now also rejects a negative value, matching src/xfer.c:2769. - The server reads its cursor from the clone card and the CloneSeqNoCard parser is gone, so there is exactly one way to set the cursor rather than two.

Aligning with canonical also corrects the cursor's meaning. Ours was the last rid sent, queried with `rid > cursor`; canonical's is the rid of the next blob to send, looped inclusively with `while( seqno<=max )`. Now that the value crosses to a real server it has to mean what canonical means, so emitCloneBatch queries `rid >= cursor` and reports the first rid it did not send, or 0 once the snapshot is exhausted. That collapses the old more/lastSentRID pair into a single nextRID and keeps the truncate hook honest by re-owing the dropped rid instead of skipping it.

TestCloneMultiBatchAgainstCanonicalServer drives the client against a stand-in server that implements the canonical contract and rejects a client-sent clone_seqno the way fossil does; it reproduces `bad command: clone_seqno` against the old client. TestHandleCloneCursorFromCloneCard pins the server half, including the inclusive cursor.

Verified against fossil 2.28 over a loopback server: fossil and sqlite now transfer every artifact (67,499 and 130,994 blobs), and wapp and pikchr still clone in 3 rounds.

* fix: conform clone cursor validation to the spec in both directions (#74)

Review caught two rules applied to the wrong side of the wire — the same axis error the original bug was.

The client rejected a negative clone_seqno as fatal, citing src/xfer.c:2769. That line is the server's rule for the SEQNO on a clone REQUEST. The reply parser is governed by §3.2 and §8.2: NEXT is recorded only when it is decimal, and a visible but non-decimal NEXT is a defined receiver-tolerance exception that leaves the recorded sequence unchanged and does not stop parsing later reply cards. Aborting the clone was the one thing the spec rules out. The rule belongs in the decoder, where a non-decimal NEXT now decodes to an unknown card: the token is preserved, no consumer records it, and following cards still parse. strconv.Atoi accepted "-1" off the wire, so this was reachable, not theoretical.

Conversely the server clamped `clone 3 0` to a cursor of 1 where §8.1 mandates a fatal: clear accumulated output, roll back, emit `error invalid clone sequence number`, and parse no later request card. Nothing is written before processDataCards, so returning early is what the rollback amounts to. Seeing this case at all needs a presence flag, because `clone` and `clone 3 0` both parsed to SeqNo 0 and only the former is exempt. §8.1 also keys on a digit-only SEQNO and withholds the fatal from wrong arity, a VERSION below 2, and a SEQNO that fails digit-only recognition, so the flag records digit-only presence rather than merely arity.

Both card rules now turn on the spec's `decimal = 1*DIGIT` production via a shared predicate instead of on whatever strconv.Atoi tolerates.

Also promotes emitCloneBatch's `count > 1` truncate guard to a documented liveness constraint. Under the old exclusive `rid > cursor` semantics `count > 0` was safe; under the inclusive cursor, dropping the only card in a batch reports a cursor equal to the one we entered with and the client re-requests forever. A check before the reply card now names a stalled round at its source instead of letting it surface as an opaque MaxRounds failure, and the cursor precondition became an error return rather than a panic — it is enforced ~350 lines away in a network-facing handler.

Records the deliberate asymmetry that our server ignores an inbound clone_seqno where canonical answers `bad command`, and renames TestHandleCloneSeqNo, which no longer sends that card.

* fix: name the clone SEQNO flag for the shape it records (#74)

HasSeqNo described presence, but the field records whether the SEQNO token is digit-only: `clone 3 -1` carries a SEQNO and sets it false. §8.1 keys its fatal on that shape rather than on the parsed value, which is why a plain int cannot stand in for it. SeqNoIsDecimal says so.

Reading it as presence would also collapse `clone 0 0` — a distinct wire form that parses to the same Version and SeqNo as a bare `clone` — back into `clone` on re-encode. encodeClone already guarded against that; a round-trip test over every clone wire form now pins it, and fails on exactly that input when the guard is removed.

Also records why the §8.1 rollback is currently vacuous: no card handled before processDataCards writes anything, so returning early leaves nothing to undo. Nothing enforces that ordering, and §3.5's ci-lock would break it, so the dependency is written down next to the code that relies on it.

Corrects a test comment that claimed a continuation property it did not exercise — the malformed card sits in the trailer, so nothing follows it. That half of §8.2 is covered by the follow-on card in TestDecode_CloneSeqNoNonDecimal. check-in: f2fa2cc751 user: [email protected] tags: trunk

* Enforce card occurrence classes and intra-run ordering

The deck parser enforced card-type ordering and nothing else. Each of the 14 single-occurrence card types was handled by a case that assigned, so a second occurrence silently overwrote the first, and the repeatable types were appended with no sort check at all. draft-fossil-artifact-format-00 §4.5.1 and §4.5.2 define both rules; implement them.

Duplicate rejection covers A, B, C, D, E, G, H, I, K, L, N, R, U and W. The guard is separate from the card-type check because that check permits equal consecutive letters. F, J and M gain a strictly-ascending check against the preceding entry, T the two-level (name, target hash) rule with the full sign-prefixed name as primary key. P and Q deliberately get neither: P order is semantic — primary parent first — and Q is unordered-repeatable. P also now appends across cards instead of replacing, which is what §4.5.1 means by repeatable. Z belongs to neither class, so it is accepted with no duplicate or last-card rule; the authoritative checksum is the trailing 35 bytes VerifyZ already verified.

All four ordering checks route through one comparator, deck.Compare, which is §4.5.3: unsigned octets, locale-independent, no normalization, no case folding. §4.5.3 exists because the same ordering underlies the delta-manifest merge walk and baseline binary search of §8.2, so a second copy would resolve a delta manifest wrongly rather than merely fail validation. The F check runs on the decoded filename, per §4.5.2.

Malformed input is reported through the error return, never a panic -- these bytes arrive from peers.

Two writer-side defects surfaced, both of which let this package emit artifacts its own parser now rejects. Marshal emitted J cards in caller-supplied order while already sorting F, M and T; it now sorts them. Marshal also sorted T by the concatenation name+target, which lets the target's leading hex digit compete with the tail of a longer name and reorders tags whose names are prefixes of one another; it now uses the two-level key. The F and M sorts and the R-card file enumeration route through deck.Compare as well, the latter per §6.3.

Closes #70

* Order T cards by decoded name and type the Z payload

Review found two ways the first pass diverged from canonical fossil in the accept/reject direction.

The T ordering key used the raw wire name. §4.7.16 says the name is escape-decoded and that the primary ordering key is the full name token including its leading sign character; manifest.c defossilizes the name before the ordering compare. Because "\s" begins 0x5C and a space is 0x20, the two forms order oppositely, so a run like

T *sym-a\sb aaaa... T *sym-a-c bbbb...

is valid, fossil accepts it, and we rejected it. Branch names may contain spaces, so this is an ordinary tag, not a corner case -- the same post-decode trap already handled for F. tagOrderKey now builds the decoded key and both the parser check and the marshaller's sort call it, so the two cannot drift apart.

The Z card accepted any payload. Accepting a Z line inside the body is right -- §4.5.1 puts Z in neither occurrence class and treats an earlier Z as content covered by the final checksum -- but §4.7.19 still fixes the payload at exactly 32 hexadecimal characters. Validating nothing left an arbitrary-byte channel in an otherwise fully typed grammar and would have stored and forwarded artifacts canonical refuses. Upper-case digits are accepted, matching fossil's validate16.

Also: the T sign prefix is now checked against {+,-,*} per §4.7.16, both because canonical checks it and because an unvalidated sign byte >= 0x80 would UTF-8-expand to two bytes when the ordering key is assembled and silently corrupt the comparison. Two stale §4.7.20 citations corrected to §4.7.16, and the two-level T test gained the prefix-name case that actually distinguishes a two-level comparison from a concatenated one.

* Cite the normative grammar for the case-insensitive Z token

The Z payload's case-insensitivity was justified by inferring it from the word "hexadecimal" in §4.7.19. §6.1 states it normatively: md5-token = 32hexdig-ci, a parse-accept grammar admitting both 0-9a-f and A-F. Cite that instead, and record the split it implies -- the structural check is case-insensitive while §6.2 keeps verification byte-exact against the lowercase form.

Also soften deck.Compare's claim that the delta-manifest merge walk relies on F ordering to the future tense parse.go already uses: the merge walk is a map overlay today with no comparator to share. check-in: 5036cae392 user: [email protected] tags: trunk

* Deltify predecessors on the commit path

Every locally-created artifact was stored whole. The delta table was in the schema and stayed empty, so a repository grew with the number of commits rather than the amount of change. blob.StoreDelta existed and worked but had no caller on the commit path; the only delta-store in use was blob.StoreDeltaRaw, from the sync receive path, for deltas that arrive pre-encoded. The write path never made a deltify decision at all.

Port canonical Fossil's content_deltify (2.28 src/content.c:852-947) as content.Deltify. Fossil deltifies backwards: a new artifact is always stored whole and its PREDECESSOR is rewritten as a delta against it, so the tip of every chain is full content and reading the newest version of a file stays a single blob load. Chain depth becomes an artifact's age in revisions rather than something that compounds, because an artifact already stored as a delta is never re-deltified.

blob.StoreDelta turned out to be the wrong primitive to wire up: it inserts a new row as a delta, whereas the policy needs to rewrite an existing row. It is left alone. content.Deltify lives in the content package because it needs Expand, which blob cannot import.

The policy is stated once, in content.Deltify, with each rule's canonical line cited: the 50-byte floors, the 75% ratio, the never-redeltify rule that bounds depth, the private-into-public refusal, and the loop break that undeltas the source and then still declines the pairing. Two call sites pass pairs, mirroring canonical's: insertMlinkRow for file content (manifest.c:1562) and Checkin for the parent manifest (checkin.c:3234). insertMlinkRow is shared by the fresh-commit and crosslink paths, so file deltification cannot drift between them.

On a 12-revision, 4-file history: 60 artifacts, 44 delta-encoded (73%), p50 chain depth 4, max 11, and 23,994 content bytes measured as sum(length(content)) -- down from 58,973 with everything whole. A repository produced this way passes fossil 2.28 test-integrity (60/60 blobs, 0 errors) and fossil rebuild, which re-derives every table by walking the chains.

* Bound the deltify ancestor walk, require a transaction, harden tests

Review follow-ups on the deltification policy.

Bound the ancestor walk. It was a bare loop with no cycle guard: a sync peer can shape a cyclic delta graph, because blob.StoreDeltaRaw records delta(rid,srcid) links from the wire without a cycle check, and an unbounded walk would spin inside the write transaction. Guard with the seen-set idiom content.walkDeltaChain already uses, plus a step cap. In the current ordering IsAvailable runs first on both rids and follows the same links under its own depth bound, so Deltify declines before the walk begins -- but that protection is incidental to a different question, so the bound belongs to the loop that needs it. Both behaviours are now pinned: the walk reports a cycle, and Deltify declines an ungrounded source.

Move both size floors ahead of the walk. Canonical returns on a sub-50-byte target (content.c:877-885) before entering the candidate loop, so it never reaches the content_undelta on :902. Running the walk first meant a small target whose successor chained back to it would inflate the source to full content and then decline anyway -- a storage regression and exactly the divergence from canonical's layout the surrounding comment claims to avoid.

Take *db.Tx instead of db.Querier. The rewrite is two statements, and a blob holding delta bytes with no delta row expands as though it were full content: silent corruption rather than a detectable error. Canonical wraps the pair in db_begin_transaction (content.c:938-941); the signature now makes that non-negotiable at compile time. Both call sites already passed a transaction.

Three tests were named for behaviours they did not pin, and each is now verified to fail when the behaviour is reverted:

- The 75% ratio test had no assertion at all; deleting the rule left it passing. It now asserts the pair is declined and stays whole. - The loop-break test could not distinguish declining from proceeding, so the undelta-then-proceed behaviour corrected in the previous commit was untested. It now asserts the target is left whole. - The manifest test checked only "at least one", so a regression from 11-of-12 to 3-of-12 passed. It now checks each index and names the manifest that stopped being encoded.

Also cover the source-side 50-byte floor, which the tiny-artifact test could not reach because it made both artifacts tiny and returned at the target check first, and assert that Undelta restores blob.size while Deltify leaves it alone (content.c:761 vs :935).

Gate the fossil-binary test on REQUIRE_FOSSIL_BIN, set in CI. A skip is invisible without -v, so a run with no fossil binary was byte-identical to a pass -- for the one criterion our own tests cannot substitute for.

Document that blob.StoreDelta is not the commit path's primitive. It inserts a new row as a delta; deltifying rewrites an existing one. It has no non-test callers and the package doc advertised it as the delta-storing entry point.

* Document what the deltify loop-safety argument rests on

Two comments, no behaviour change.

The IsAvailable(srcRid) guard is ours, not canonical's: content.c guards only content_is_available(rid) and then calls content_get(srcid) unguarded. It is also load-bearing beyond phantoms, because it screens out the cyclic delta graphs that would otherwise reach the ancestor walk. Say both, at the guard.

And say at the walk why its seen-set never fires in practice, so that someone reordering the prologue meets the reason rather than concluding the bound is dead code. The note lived only in the test file, which is not where that reader is looking. check-in: 2368cad91a user: [email protected] tags: trunk

* cli: add repo serve command wiring the xfer HTTP server

Repo.ServeHTTP has always been public, but nothing in the CLI called it, so a repo created or cloned with this tool could only ever be a clone leaf. Add `libfossil repo serve` as a thin adapter: open the repo, build the bind address, translate process interrupt into the context cancellation ServeHTTP already honors, and call it. No new server or shutdown logic.

Binds to 127.0.0.1:8080 by default; --addr overrides host and/or port. Verified against a stock fossil client (canonical binary, not just this library's own transport) cloning over an actual TCP round trip.

* cli: address review nits on serve interop test

Document why TestRepoServeCmdRun and TestRepoServeCmdCanonicalFossilClone are allowed to send a real SIGINT to the test process (no t.Parallel(), no competing signal handlers in package cli_test), so a future test in this package doesn't silently break that assumption.

Also log the canonical fossil binary path and version the interop test actually exercised. A skip and a pass read identically in a non-verbose CI log; if the apt install step that puts fossil on PATH is ever removed or reordered, this line is what would tell us the test stopped proving anything instead of just staying green. check-in: 7572f7d420 user: [email protected] tags: trunk

* fix: distinguish absent, empty, and valued U-cards in manifest parsing

Manifest parsing collapsed a wholly absent U-card and a present-but-empty U-card into the same stored value (empty string), so every crosslink call site that wrote the check-in user into event/attachment/forumpost rows lost the distinction between "no login recorded" (should be SQL NULL) and "login explicitly blanked" (canonical fossil resolves this to the literal "anonymous" at parse time — see src/manifest.c:1008-1016).

Change deck.Deck.U from string to *string so the three states are representable: nil (absent), non-nil "anonymous" (present-empty, resolved in parseUCard the same way fossil's own parser does), and non-nil with a value (present). Resolving once in the parser means every downstream crosslink/rebuild call site can keep binding d.U directly — Go's database/sql converts a nil *string argument to SQL NULL and dereferences a non-nil one automatically, so no per-call-site special casing is needed. The type change itself makes the old ambiguous string-binding path a compile error everywhere Deck.U is constructed, closing off the drift that caused issue #29.

Added deck.Str, a small pointer-literal helper, since Go can't take the address of a string literal directly; existing tests that built Deck{U: "..."} now go through it.

* fix: review nits — whitespace U-cards, rebuild coverage, tag.AddTag, naming

Four follow-ups from review of the U-card presence fix:

- deck.parseUCard now trims args before the emptiness check, matching canonical fossil's next_token() (skips leading whitespace) and every sibling card parser in this file. Previously "U \n" stored literal whitespace instead of resolving to "anonymous". Added a whitespace-only case to TestParseUCardStates.

- Added TestRebuild_UCardPresenceStates covering the rebuild path (rebuild_manifest.go's rebuildCheckin), mirroring TestCrosslink_UCardPresenceStates. The binding site was already correct (same nil-to-NULL pointer semantics as crosslink.go) but had no dedicated regression test.

- tag.AddTag now panics on an empty opts.User, matching the panic idiom CheckinOpts.User already establishes. Previously a tag created with an empty User silently crosslinked to user=''; that asymmetry with CheckinOpts was a wart worth closing rather than documenting as intentional.

- Renamed deck.Str to deck.User: the name should say what the helper constructs (a Deck.U value), not how it's implemented (a pointer take). check-in: 527759acd5 user: [email protected] tags: trunk

* fix: store delta content against a phantomized base instead of discarding it

A delta artifact arriving before its base is normal steady-state behavior during a Fossil transfer, not a fault. The receive path used to discard such deltas outright and phantomize the target instead, so a phantom only ever cleared when its base happened to already be present -- resolution advanced one delta-chain hop per round, making clones of deep-history repos provably unable to converge within MaxRounds.

Match canonical Fossil's content_put_ex/rid_from_uuid model: resolve the delta's source via a create-phantom-if-missing lookup, persist the delta bytes unconditionally (reading the target size straight out of the delta header, so the base never needs to be readable to store it), and record the delta-to-source link immediately. Availability resolves lazily via the existing content.IsAvailable chain walk and manifest.Crosslink's deferred retry -- both already correct and untouched here.

storeReceivedFile is the single call site shared by clone, sync-pull, and server-push, so fixing it here also closes out the delta-before-base sentinel handling on the pull and push paths (issue #57, folded into this fix): neither path needs to special-case the condition anymore because it stops being an error.

Verified against the pikchr corpus via a loopback fossil server: rounds dropped from 100 (exhausted) to 3, files received from 63,155 (22.2x) to 2,980 (1.05x), and the phantom table empties out completely.

* fix: converge delta storage on canonical, close integrity gap, harden wire parsing

Three defects from review, all in the raw-delta path added for the delta-before-base fix, all reachable from an untrusted peer:

1. delta.getInt accumulated v = v*64 + digit with no overflow check. A twelve-byte header of maximum-value digits wraps to exactly math.MaxUint64, which casts to int64(-1) -- the same sentinel blob.size uses for "phantom". That silently corrupted the row: real content reporting as a phantom, with no phantom-table entry to make it re-requestable. getInt now rejects overflow instead of wrapping, and parseTargetLen (shared by Apply and OutputSize) bounds the parsed target length to a sane maximum before it ever reaches an allocation or a size column.

2. StoreDeltaRaw panicked on a zero-length delta payload. storeReceivedFile guarded payload == nil but not len(payload) == 0, and nothing between wire decode and storage checked for it -- a peer sending an empty delta file card crashed the process. Wire data gets an error return, not an assertion; OutputSize and StoreDeltaRaw are now consistent with delta.Apply's existing "empty delta is an error" contract.

3. The receive-time split (store raw only when the base happens to be unavailable, otherwise eagerly expand and verify) left a gap an attacker chooses the timing of: withhold the base, and a delta claiming any UUID goes down the unverified raw path. Nothing downstream checked it either -- content.Expand applied deltas with no hash check, and content.Verify was reachable only from repo rebuild and one integration test.

Closed by converging fully on canonical's model: every delta is now stored raw and linked via a create-phantom-if-missing lookup, regardless of whether its base is currently available -- receive time no longer branches on availability at all. Verification moves to the one choke point every expansion path already goes through: content.Expand now hashes its result against the row's claimed UUID before returning it, catching a mismatch regardless of which caller or how the chain assembled. content.Verify becomes a thin wrapper over the same check.

Storage-layer and end-to-end regression tests added for all three, including the exact claim-one-UUID/ship-a-delta-for-another/withhold-the- base reproduction, guarded against degenerating into a no-op (delta must actually compress, base must be genuinely absent, rejection must be a real error rather than merely different bytes).

Re-verified against pikchr (2,842 artifacts) and wapp (1,039): pikchr converges in 3 rounds, 2,980 files, 0 phantoms; wapp converges in 3 rounds, 1,115 files, 1 phantom outstanding (pre-existing, unrelated -- clone.go has no DB-phantom rescan the way the pull path's loadDBPhantoms does; separately tracked). On-disk repository size is close to but slightly smaller than storing everything as raw delta bytes should produce (pikchr 6.81MB -> 6.73MB, wapp 5.64MB -> 5.46MB): zlib on full text already captures most of the redundancy an explicit delta would, so the win here is mainly integrity and convergence, not size.

* fix: bound delta allocation to actual work, validate delta source hex

Two defects from round-two review, plus a stray comment.

1. delta.Apply preallocated its output buffer directly from the delta header's claimed target length (make([]byte, 0, targetLen)). That claim only had to clear maxDeltaTargetSize (4 GiB) to be trusted as an allocation size, so a handful of wire bytes declaring a near-4 GiB target triggered an allocation sized to the claim, not to any work actually done. Harmless as a one-off parse spike; durable once a delta can be stored unverified (see blob.StoreDeltaRaw, this branch) -- a hostile peer plants one such delta and every later Expand of that row repeats the attempt. The initial capacity is now a small fixed bound; append grows it normally for legitimately large content instead of trusting the header's own claim about itself.

2. storeReceivedFile validated the target UUID's hex-ness (hash.IsValidHash) but not the delta source's. The blob table's CHECK constraint only bounds uuid length, not hex-ness, so a non-hex string of valid length passed straight through to blob.StorePhantom and became a permanent, unfillable row -- one the pull path's loadDBPhantoms would then re-request every sync round, forever. deltaSrc now gets the same check as the target uuid, at the same place.

Also deleted a stray doc-comment line left over from an earlier refactor.

Regression tests for both, RED-verified against reverted code before being fixed: the allocation test had to be corrected mid-way through -- delta headers encode their integer in Fossil's own base-64 alphabet, not decimal, so an initial fmt.Sprintf("%d", ...) fixture parsed as a different (still large but differently-bounded) value and passed vacuously; appendInt is the real encoder and is used directly now, with a round-trip sanity check against OutputSize guarding the fixture itself. check-in: 89535a6c42 user: [email protected] tags: trunk

* checkout: re-stat executable bit at commit time, not just on changed files

Change scanning compares content hashes, so a file whose only edit is a chmod (owner-execute bit gained or lost, no content change) is never flagged as changed. vfile.isexe then carries forward whatever was captured at the last Add/Commit, and the resulting F-card silently keeps the stale permission.

Mirror canonical Fossil's belt-and-braces approach (src/checkin.c): manifest generation re-stats each committed file's live on-disk mode independently of whatever change scanning recorded, using the existing modeIsExecutable predicate (added for the prior executable-bit fix) rather than trusting a cached flag a second time. This runs for every file with a vfile row that is part of the commit, not only ones flagged as content-changed, so a chmod-only edit is now captured.

Applies only to files actually present in vfile, leaves symlink markers alone, and is a no-op on Windows where os.FileMode carries no owner-execute bit.

* checkout: skip missing files during commit-time perm re-stat

restatCommitPerms Stats every tracked file that is part of the commit, and a Stat failure on any of them aborted the whole commit. That regressed a case the mode-change fix never meant to touch: a tracked file deleted from disk without Unmanage, with an unrelated file modified, used to commit successfully and carry the missing file's last-committed content forward — after the fix it failed the commit outright, on the implicit "commit everything" path in particular.

Skip only os.IsNotExist and carry the existing Perm forward unchanged, matching pre-fix behavior exactly. This fix is about mode changes being missed, not about detecting stale tracking — that is a legitimate feature canonical Fossil also checks for at commit time, but it deserves its own issue rather than arriving as a side effect here. Any other Stat error (permission denied, I/O fault, ...) still fails the commit. check-in: 094abd4b4e user: [email protected] tags: trunk

* fix: give Timeline its own result type so an Ancestry cursor cannot compile into it

LogEntry (Ancestry's result type) carried a Cursor field that was always the zero value on Ancestry entries — structurally identical to a legitimate Timeline first-page call. Feeding it into TimelineOpts.After silently paginated from page one forever with no error; documenting the field as invalid on Ancestry entries did nothing to stop it, because the zero cursor cannot be told apart from a valid first-call state even in principle.

Split the shared type instead of adding a runtime guard: LogEntry loses Cursor entirely and remains Ancestry's result type, while a new TimelineEntry (embeds LogEntry, adds Cursor) becomes Timeline's result type. An Ancestry entry's cursor can no longer be obtained at all, so passing one to Timeline is now a compile error rather than a documented hazard. Applied the same split to the internal manifest.LogEntry / manifest.TimelineEntry pair Timeline and Ancestry are built on.

Breaking change; CHANGELOG documents the migration.

* doc: warn against hand-assembling TimelineEntry from an Ancestry LogEntry

Review found the one remaining escape hatch: TimelineEntry{LogEntry: someAncestryEntry} compiles with a zero-value Cursor, which Timeline reads as "start from newest" rather than an error. No library path produces that value — it requires deliberately embedding an Ancestry LogEntry into a hand-built TimelineEntry — but flag it at the type so whoever does that gets the warning where they're typing. check-in: ad09ba3d97 user: [email protected] tags: trunk

Both were accepted by the public CloneOpts but never forwarded to the internal clone path, so setting either silently did nothing. The internal clone options struct has no matching fields at all, so wiring them would mean designing semantics first — is a caller- supplied code a validation assertion or an identity override? — and no caller has asked for either. Delete them rather than guess.

CloneResult.ProjectCode and CloneResult.ServerCode are unrelated and unaffected: they are populated from the internal clone's return value (the remote, via protocol negotiation), not from caller options, and continue to work exactly as before. Consistent with the same call made on UpdateOpts.Force. check-in: 9776c25462 user: [email protected] tags: trunk

Force was accepted by the public UpdateOpts but never forwarded to the internal update path, so setting it silently did nothing. Delete it rather than guess at forcing semantics; a real implementation can be designed and added later as a deliberate, wired field if a caller needs it. Rides along with the UpdateResult signature change already landing on Update in this cycle. check-in: cd674f058b user: [email protected] tags: trunk

* fix: populate mlink parent columns on the Crosslink check-in path

insertCheckinMlinks (the xfer/Crosslink ingestion path) wrote mlink rows with only (mid, fid, fnid) set, leaving pmid/pid NULL. Fossil's compute_fileage filters mlink.fid != mlink.pid, and in SQLite <int> != NULL evaluates to NULL rather than true, so every row written by this path was silently dropped from ancestry-walk queries - `fossil ls -R <repo> -r <rev>` returned empty for any commit that arrived over the wire, even though `fossil open` worked fine (it reads the manifest blob directly and never consults mlink).

Root cause: the two check-in paths that write mlink both needed to implement the same three-case pid convention (src/manifest.c:1668-1679 in canonical Fossil - carried-over/added/added-by-merge) and had drifted. insertMlinks (the direct check-in path) only handled two of the three cases and ignored merge parents entirely; insertCheckinMlinks didn't populate the parent columns at all.

Factor the convention into one place (internal/manifest/mlink.go): resolveMlinkParent implements the pid/pmid rule, insertMlinkRow wraps it and also fills pfnid (rename tracking) and mperm (permission encoding), which neither path wrote before. Both insertMlinks and insertCheckinMlinks now call insertMlinkRow, so the two write paths produce indistinguishable rows and cannot drift apart again.

insertCheckinMlinks also stopped skipping F-cards with an empty UUID (the delta-manifest deletion marker) - it now writes a row with fid=0, matching canonical's "deleted files have mlink.fid=0" and letting deletions participate in ancestry queries the same way Fossil's own writer does.

Un-gates ls_branch_tip_repro_test.go (previously untracked, guarded behind LIBFOSSIL_LS_BRANCH_TIP_REPRO) as a permanent regression test, and adds explicit coverage in internal/manifest/mlink_test.go for all three pid cases (normal add, merge add, delete) on both write paths.

* test: pin the deleted-file pid to the parent's exact file rid

Strengthens the deleted_file_gets_fid_zero subtest added for #29: it was only checking pid != 0, which would pass even if resolveMlinkParent picked the wrong nonzero value. Compare against the primary parent's actual mlink.fid for the same filename instead.

* fix: review findings on #29 — substring perm match, test isolation

Blocking: permToMperm switched on the whole perm string, matching only exact "x"/"l". Canonical manifest_file_mperm (src/manifest.c:1482-1492) does a substring test (strstr), and F-card perm fields can carry more than one character — internal/deck/parse.go:194 assigns the field verbatim from remote xfer input, and Fossil itself emits multi-char perm strings (the " w" rename placeholder, see #51). An exact match on e.g. "wx" would silently drop the executable bit, the exact invariant PR #48 landed to protect. Switch to strings.Contains, x checked before l to match canonical's order (the same idiom already used at internal/checkout/vfile.go:52). Added TestPermToMperm_SubstringMatch, including a "wx" case pinned to mperm=1.

Test isolation: rowFor in both mlink_test.go table-driven tests closed over the parent *testing.T, so a Fatalf in one subtest's row lookup aborted every subtest after it via runtime.Goexit unwinding the shared parent goroutine. Pass each subtest's own t into rowFor instead. Proven on the reviewer's revert run: after the deletion subtest's Fatalf, the merge-add and normal-add subtests never even started.

Tightened the carried-over-file assertion in TestInsertMlinks_MergeParentsGetPidNegativeOne from "pid != 0" (would pass for -1 too) to pin the exact parent fid, matching how the crosslink-path twin test already asserts it.

Documented (comment only, no behavior change) a known divergence from canonical in resolveMlinkParent: a file present in a merge parent with different content (a conflict resolved during the merge) should stay pid=0 in canonical Fossil, but this function returns pid=-1 for any file present in a merge parent regardless of content match. Low blast radius — every mlink consumer in this package filters pid != fid, which holds either way — but worth flagging so it isn't rediscovered as a surprise. check-in: 24c35dad87 user: [email protected] tags: trunk

* checkout: Update returns UpdateResult so conflicts are visible

The internal 3-way merge already tracked which files were written, removed, and left with conflict markers, but Checkout.Update discarded all of it and returned a bare error. That collapsed a clean update and one that silently wrote <<<<<<< markers into working-tree files into the same outcome, with no way for a caller to tell them apart.

Update now returns (UpdateResult, error). UpdateResult.Conflicted lists paths that were merged but not cleanly -- this is still a successful update (err == nil), not an error, since the working tree is usable and the caller just needs to know which files to show a user. A genuine failure still returns a non-nil error paired with a zero-value UpdateResult. FilesWritten/FilesRemoved are populated the same way, as paths rather than counts.

Threading the paths up required changing the internal update's processFileUpdates from returning bare int counts to returning the actual []string paths, since the count was all that existed before. Checkout.Extract is untouched -- it has no conflict concept.

Breaking change, clean break at v0.7.0 (pre-1.0).

* checkout: sort Update result paths, document FilesWritten/Conflicted overlap

Update's result slices were built by ranging over maps.allNames, a map[string]bool, so their order was whatever Go's randomized map iteration happened to produce that run. A caller diffing result.FilesWritten against a golden file, or asserting on it with reflect.DeepEqual, would see it fail intermittently for no reason related to their code.

Sort all three slices (sort.Strings) right before Update returns. Added a test that touches four files -- two conflicting, two clean additions, named so map iteration order is very unlikely to hand them back sorted by luck -- and asserts the exact sorted slice on both FilesWritten and Conflicted. Verified it fails reliably (5/5 runs) with the sort calls temporarily removed, then restored the fix.

Also stated outright, in both the internal and public UpdateResult doc comments, what "clean or not" was only implying: every path in Conflicted also appears in FilesWritten, since the conflict markers are what got written there. check-in: 75f3521e54 user: [email protected] tags: trunk

Repo.Timeline was an ancestry walk wearing an enumeration's name: it required a start rid, emitted that check-in, followed plink where isprim=1 to the primary parent, and repeated until the chain ended. It never touched the event table and never visited a second parent or a sibling branch head, so it silently omitted any check-in that wasn't a first-parent ancestor of the given start rid — on libfossil's own repository that was 577 of 3,761 check-ins, with no error and no warning. The limit option never bound because the (wrong) walk exhausted first.

Split into two operations under honest names:

- Repo.Timeline(TimelineOpts) enumerates the event table newest first, every kind by default or a single kind via Type. This matches canonical `fossil timeline`'s -t/--type default (a filter only applies when given and not 'a'). - Repo.Ancestry(LogOpts) keeps the original first-parent walk, unchanged in behavior, under a name that says what it does.

Breaking change, no deprecated shim (pre-1.0, clean break): callers of the old Timeline(LogOpts) need to move to Ancestry(LogOpts) if they want the walk, or to Timeline(TimelineOpts) if they want the enumeration fossil's own timeline command performs. The only in-tree callers were the `repo timeline` CLI command and the two examples; all three are migrated to Timeline(TimelineOpts). See CHANGELOG.md.

LogEntry gains a Kind field identifying which of event.type's six kinds (ci, e, f, g, t, w) an entry is; Parents is only populated for check-ins, since plink only relates check-in artifacts.

Timeline's pagination cursor orders by (mtime DESC, rid DESC) instead of canonical fossil's bare mtime DESC with no tie-break — a deliberate improvement, not an oversight, since fossil's web cursor combines that missing tie-break with a one-second timestamp slop and can repeat or skip rows at a page boundary. Getting the composite cursor to actually round-trip losslessly required fixing a real bug along the way: JulianToTime truncated instead of rounded when converting a julian day back to milliseconds, so TimeToJulian(JulianToTime(m)) was not always equal to m even for exact-millisecond inputs — enough to misplace a row at a page boundary under exactly the same-mtime conditions this issue's acceptance criteria test for. Fixed to round to nearest. Because the same class of drift also appears in the ncruces/WASM sqlite driver's own internal REAL-to-time.Time conversion (independent of this library's code), the cursor comparison also carries a small explicit tolerance rather than relying on bit-exact equality across drivers.

Both the ancestry walk's loop and the enumeration's row loop now carry an explicit, testable bound so a corrupt or cyclic plink table can't hang the process. check-in: 9a2697c556 user: [email protected] tags: trunk

* fix: separate blob existence from blob availability (#43)

A phantom is a real blob row with size = -1 and NULL content, so blob.Exists — "SELECT rid FROM blob WHERE uuid=?" — answers true for it. Every guard built on Exists therefore admitted artifacts whose content cannot be read, and content.Expand aborted with "rid N is a phantom".

Canonical Fossil keeps two predicates and never substitutes one for the other: rid_from_uuid (existence, xfer.c:70) and content_is_available (availability, content.c:163). The latter is transitive — it follows delta.srcid up the chain, so a delta whose base is a phantom is unavailable even though every row in the chain exists.

Add content.IsAvailable / content.AvailableByUUID, ported from content_is_available including the chain walk, with an explicit depth bound so a cyclic delta table terminates instead of hanging. The walk lives in exactly one place; no caller touches the delta table.

blob.Exists keeps its behavior and gains a doc comment stating that it answers existence only, returns true for phantoms, and naming the availability predicate as the alternative.

manifest.blobPresent is deleted. It was a local patch at one call site that tested size >= 0 without the chain walk, so it still reported a delta-on-phantom as present — the same class of bug it was written to work around.

Every blob.Exists call site was classified as existence or availability and converted where it guards a content read. Sites that want rid identity — plink/mlink/tagxref population, cluster phantom creation, private-status bookkeeping, the post-store sanity check in handleFile, and phantom filling in storeReceivedFile — keep existence, since delta-before-base is a normal steady state during clone.

Verified against a loopback fossil 2.28 server: the 23-artifact repro that previously failed roughly half the time now clones 6 of 6.

* fix: IsAvailable must fail closed when the delta lookup errors

Only sql.ErrNoRows means the chain is grounded. Any other failure — cancelled context, closed connection, NULL srcid — says nothing about groundedness, so returning true there claims readable content that was never verified. That is the same class of bug this predicate exists to prevent, and it runs on data arriving over the wire during clone.

The blob-size lookup above already fails closed on the same pattern; this restores the symmetry. check-in: 670ca0e75f user: [email protected] tags: trunk

Manage() inserted new vfile rows with isexe always defaulting to 0, regardless of the file's on-disk permissions. buildCommitFiles() also never consulted vfile.isexe when constructing the manifest F-card for changed/added files — it only carried forward the parent manifest's Perm string, which is empty for a brand new file. Together this meant an executable file added via Manage always committed without an "x" marker, and a later Extract wrote it back as 0644.

Fix: - Manage() now stats the file and records isexe using the same predicate as Fossil's file_perm() (src/file.c:316): owner-execute on a regular file (S_ISREG && S_IXUSR). Group/other-only execute (e.g. 0645) does not count, matching canonical behavior. The detection lives in one place (modeIsExecutable, next to the existing isexe-to-mode helper vfilePerm) rather than being duplicated at call sites. - buildCommitFiles() now derives the F-card Perm for changed/added files from vfile.isexe instead of only inheriting the parent's Perm, without disturbing an existing symlink marker ("l").

Extract already read isexe correctly when writing files back to disk, so no change was needed there.

Out of scope, left untouched: symlink (islink) handling, and executable-bit CHANGE detection on an already-tracked file whose mode changes without going through Manage again (Fossil's belt-and-braces re-stat at commit time / VFileIsExec path) — that path appears to still be missing here and should be tracked separately if it matters. check-in: 76dd5241c2 user: [email protected] tags: trunk

Fossil's check-in manifests require only the D and Z cards; the U-card (user) and C-card (comment) are optional, and canonical fossil binds a missing U-card as SQL NULL on the event row. manifest.Log scanned both columns into plain strings, so any repository containing such a check-in failed the entire walk with "converting NULL to string is unsupported" and returned zero rows.

Scan event.user and event.comment through sql.NullString and use its zero-value String field, so a NULL column becomes "" without changing LogEntry's public shape - no nullable type leaks out, and no caller has to know NULL was ever possible. Non-NULL columns are unaffected.

repo timeline now renders an empty user as "?", matching canonical fossil's coalesce(euser,user,'?') TTY output; the substitution stays in the CLI's presentation layer, not the library.

Fixes #44 check-in: 9564ccad55 user: [email protected] tags: trunk

Newly created repositories were missing the ticket and ticketchng tables and the ticketchng_idx1 index that fossil new provisions. Stock Fossil's web UI queries the ticket table unconditionally (e.g. info.c's artifact-detail lookup: SELECT 1 FROM ticket WHERE tkt_uuid GLOB ...), so opening a libfossil-created repository with the stock fossil binary and visiting /info/<uuid> crashed with "no such table: ticket".

Add the ticket and ticketchng table definitions and the ticketchng_idx1 index to schemaRepo2, taken verbatim from zRepositorySchema2 in fossil's src/schema.c, in the same position canonical fossil places them (after attachment_idx2, before cherrypick).

Verified against a real fossil new repository: sqlite_master table/ index diff and per-table .schema diff for ticket/ticketchng now match exactly. check-in: 099ddb1b7a user: [email protected] tags: trunk

Bumps the go_modules group with 1 update in the / directory: [golang.org/x/crypto](https://github.com/golang/crypto). Bumps the go_modules group with 1 update in the /observer/otel directory: [golang.org/x/crypto](https://github.com/golang/crypto).

Updates `golang.org/x/crypto` from 0.49.0 to 0.52.0 - [Commits](https://github.com/golang/crypto/compare/v0.49.0...v0.52.0)

Updates `golang.org/x/crypto` from 0.49.0 to 0.52.0 - [Commits](https://github.com/golang/crypto/compare/v0.49.0...v0.52.0)

--- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-version: 0.52.0 dependency-type: direct:production dependency-group: go_modules - dependency-name: golang.org/x/crypto dependency-version: 0.52.0 dependency-type: indirect dependency-group: go_modules ...

Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> check-in: c5e189600f user: [email protected] tags: trunk

* docs: rebrand project to go-libfossil; add acknowledgements

Rename the project/repo identity to go-libfossil while keeping the Go module path (github.com/danmestas/libfossil) and the libfossil CLI/package names unchanged — imports and installs are unaffected, GitHub redirects cover the renamed repo URLs.

- README: title, Tests badge repo URL, and a note clarifying the repo-name vs import-path split. - Add an Acknowledgements section crediting Stephan Beal (libfossil), Dr. D. Richard Hipp / Fossil SCM, TigerBeetle TigerStyle, John Ousterhout (A Philosophy of Software Design), and Antithesis (DST), with links. - Docs site: hugo.yaml title + GitHub repo links, landing page identity surfaces. baseURL, module paths, and CLI/API names left intact.

* docs: add go-libfossil logo to README check-in: 97ae7eb00d user: [email protected] tags: trunk

Two concurrent libfossil.Repo handles on the same SQLite file fail their writes with SQLITE_BUSY immediately, well under the configured busy_timeout=5000 window. Reproduces ~50% of the time on contended manifest.Checkin paths.

Root cause: db.WithTx opens BEGIN DEFERRED. Under WAL, a deferred transaction acquires SHARED on first read, then upgrades SHARED→RESERVED on first write. If two connections race that upgrade, SQLite's deadlock-avoidance returns SQLITE_BUSY *immediately*, bypassing busy_timeout entirely (documented behavior, sqlite.org/c3ref/busy_timeout.html).

Fix: prepend _txlock=immediate to the DSN in both modernc and ncruces drivers. Every BEGIN now acquires RESERVED upfront, so the second writer blocks at BEGIN — where busy_timeout *does* apply — and the two transactions serialize cleanly.

Regression tests added to both driver packages: - TestBuildDSN_TxlockImmediate covers the DSN shape across empty/non-empty pragma maps. - TestConcurrentWritersDoNotRace reproduces the issue-#33 contention pattern (two conns, SELECT then INSERT under DEFERRED) over 25 iterations and asserts zero SQLITE_BUSY. Without the fix, this test fails 25/25; with the fix, 0/25.

Closes #33. check-in: 202576c58c user: [email protected] tags: trunk, v0.6.3, db/driver/ncruces/v0.2.0, db/driver/modernc/v0.2.0

Repo.Commit was a thin pass-through to manifest.Checkin: callers who supplied a subset of the parent's tracked files got a partial manifest, so files tracked at the parent silently disappeared from the new commit. A checkout at the child's rev saw only the supplied subset — bones, EdgeSync, and sesh all hit this as silent data loss across slot commits.

Default behavior now matches `fossil ci`: when ParentID > 0 the parent's F-cards are merged into the commit's file set (caller wins on name collision; untouched parent files are loaded by UUID and carried forward).

For the rare case where a caller intentionally wants omission to mean deletion, CommitOpts.PartialManifest opts out of the merge. The internal diff/deletion tests that previously relied on the partial behavior now set this flag explicitly. A first-class deletion API may replace the escape hatch later.

Adds TestCommitPreservesParentFiles as the regression: parent [a,b,c], child supplies only [b'], child's ListFiles must return [a,b',c] with a and c keeping the parent's blob UUIDs. check-in: 862dcb20c3 user: [email protected] tags: trunk, v0.6.2

* feat: CreateOpts.ProjectCode — accept explicit project-code on Create

Downstream EdgeSync coordinates a shared project-code across sibling Fossil repos created on different machines so its NATS sync subject ("<prefix>.<project-code>.sync") matches across leaves. Today every Create generates a fresh project-code, so commits never propagate.

Add CreateOpts.ProjectCode: empty preserves current behavior; non-empty must be 40-char lowercase hex (^[0-9a-f]{40}$) — the same format generated project-codes use. Validation runs at the top of libfossil.Create before any disk I/O, so invalid input cannot leave a half-written file behind. Server-code stays auto-generated.

Threaded through repo.Create / repo.CreateWithEnv / db.SeedConfig. SeedConfig validates as defense-in-depth (public package, multiple callers possible).

Clone accepting ProjectCode is out of scope; will follow separately if EdgeSync needs it.

* test: round-trip explicit ProjectCode through fossil rebuild

Asserts that vanilla `fossil` (2.28) accepts a Go-side repo created with an explicit ProjectCode, and that `fossil sql` reads back the same value we set. Closes the gap where the existing TestCreate_FossilValidation only covered the generated-project-code path. check-in: 6b66ea6b2d user: [email protected] tags: trunk, v0.6.1

Closes #27.

libfossil opens repos in WAL journal mode but the Close chain never ran wal_checkpoint, so closed repo files were a 4 KiB stub plus a fat -wal sidecar. Vanilla fossil's validity probe stats the main file before consulting the WAL → repo rejected as "not a valid repository".

- Adds a typed CheckpointMode (Passive / Full / Restart / Truncate) and a Checkpoint(mode) method on the public Repo (mirrored through internal repo and db.DB). CheckpointMode and its constants are re-exported from the package root so callers don't have to import db. - db.DB.Close now runs PRAGMA wal_checkpoint(TRUNCATE) before closing the underlying *sql.DB. Checkpoint errors are joined with the close error via errors.Join so the connection is always closed regardless of checkpoint outcome. - Skips the close-time checkpoint on the WASM/WASI build path where WAL isn't used. check-in: faf57ad9c7 user: [email protected] tags: trunk, v0.6.0

Repo.Diff has a multi-file return shape ([]DiffEntry) but previously rejected filePath == "" with "filePath is required". Consumers that wanted a whole- checkin diff (the equivalent of `git diff revA..revB` or `fossil diff --from revA --to revB`) had to enumerate changed files themselves and call Diff per file; EdgeSync's Agent.Diff is the in-tree example of that workaround.

This relaxes the guard: when filePath == "", Diff now returns one DiffEntry per file that changed between ridA and ridB, sorted by Name for deterministic ordering. The non-empty filePath path is unchanged. Identical checkins return ([]DiffEntry{}, nil).

Whole-checkin enumeration is name-keyed (union of files across both manifests where the content UUID differs or the file exists on only one side). Proper rename detection via mlink (so a pure rename produces a single entry rather than an add+delete pair) and permission-only change reporting are deferred to a follow-up; the limitation is documented on Diff's godoc and called out in the PR description. The single-file path is unaffected.

The previous TestDiff_EmptyFilePath (which asserted the rejection error) is replaced with positive coverage for identical checkins, single-file change, multi-file change, addition-only, and deletion-only.

closes #24 check-in: 764a82ecdb user: [email protected] tags: trunk, v0.5.0

Consumers (bones, EdgeSync) currently bypass the methods-only Repo API to convert RIDs to UUIDs by reaching into r.DB() and running raw SQL. The lookup is well-defined and recurs throughout the libfossil internals; expose it as a first-class method.

Add Repo.UUIDFromRID(rid int64) (string, error) backed by a small internal helper Repo.UUIDByRID on internal/repo so the SELECT lives in one place. Missing rids surface ErrArtifactNotFound (wraps sql.ErrNoRows behind a typed sentinel) with the rid in the message.

closes #23 check-in: 61aa4dd022 user: [email protected] tags: trunk

* docs: add cross-repo release automation design spec

Designs an automated release pipeline across libfossil → edgesync → bones: - repository_dispatch chain replaces manual go.mod bumps - linear-only graph (libfossil does not dispatch directly to bones) - per-repo `make ci` mirrors GitHub CI verbatim; `make ci-fast` for pre-push - `make release VERSION=...` helper with annotated tag template - GoReleaser auto-changelog (edgesync, bones); `gh release --generate-notes` for libfossil - 6-PR rollout: 3 hygiene PRs first, then 3 dispatch wiring PRs

See docs/superpowers/specs/2026-04-30-cross-repo-release-automation-design.md

* docs: add cross-repo release automation implementation plan

Breaks the design spec into 6 sequential PRs across libfossil, edgesync, and bones — 3 hygiene PRs (Makefile + pre-push hook + release helper) followed by 3 dispatch-wiring PRs.

Plan includes: - Phase 0 prerequisites (DISPATCH_PAT setup per upstream repo) - Per-PR branch creation, file-level steps, expected commands/outputs, commit messages, and PR-creation gh commands - Post-merge smoke tests for each receiver/sender combo - Final end-to-end cascade validation via v0.4.6-rc1 tag

See docs/superpowers/plans/2026-04-30-cross-repo-release-automation.md

* docs: correct bones make ci scope to include make check

bones' .github/workflows/ci.yml runs make check (fmt-check + vet + lint + race + todo-check) before the otel-tagged lanes. The earlier scope captured only the otel lines, missing the discipline suite that is the primary CI gate. Update both spec and plan to match the actual workflow.

The cross-repo leaf-binary integration step (sibling EdgeSync + make leaf) remains intentionally CI-only — it requires a sibling working tree that doesn't generalize to a single-repo make target.

* docs: capture libfossil deviation (no pre-push, no ci-fast)

PR #3 (#19, merged 2026-04-30) shipped a tighter scope than PR #1/#2 because libfossil's existing pre-commit is already comprehensive. Document the option (A) decision in both spec and plan, and add a multi-module note for the future case if pre-push is ever added.

* docs: add go get retry + actor field to receiver workflows

PR #4 (bones #90) added a follow-up commit per code review with these two improvements; capture them in spec/plan so PR #5's edgesync receiver picks them up automatically:

- 3-attempt retry with 15s backoff covers GOPROXY propagation race (dispatcher fires immediately on tag push, receiver may outrun the module proxy). workflow_dispatch fallback still exists for manual recovery. - Actor field plumbed payload→input→github.actor with triple fallback and rendered as 'Triggered by: @actor' in bump PR body for audit. check-in: 339643c721 user: [email protected] tags: trunk

On tag push (v*): 1. Create GitHub release with auto-generated notes from PRs/commits since previous tag. Source tarball auto-attached. 2. Dispatch repository_dispatch event to danmestas/EdgeSync to trigger a chore-bump PR there.

libfossil is a library — no binaries are published, so goreleaser is overkill. 'gh release create --generate-notes' provides equivalent auto-changelog and source archives in 5 lines.

Closes the cascade chain. Once merged, the next libfossil tag will fire PRs in EdgeSync (PR #5 receiver) and via that, in bones (PR #4 receiver). check-in: 38e4948972 user: [email protected] tags: trunk

* feat(make): add ci targets mirroring test.yml verbatim

ci aggregates 3 sub-targets matching the 3 GitHub Actions jobs: ci-default — modernc driver tests + DST + vet + cmd build ci-ncruces — ncruces driver tests with -tags test_ncruces ci-otel-target — observer/otel sub-module tests

All use GOWORK=off (matching CI). Existing test/test-drivers/test-otel targets are left untouched — they don't use GOWORK=off and have drifted from CI; ci-* are the single source of truth going forward.

No ci-fast / pre-push hook in this PR — libfossil's existing pre-commit hook (modernc + ncruces + vet + otel + build + SDK drift, ~45s) already exceeds what ci-fast would provide. Adding a pre-push would just double push cost without catching new bugs.

* feat(make): add release VERSION=... helper

Validates clean tree, runs make ci, opens \$EDITOR with auto-populated tag annotation template, creates annotated tag locally. Push is intentionally manual.

* feat: add release annotation template

Used by 'make release VERSION=...' as the body skeleton for annotated tags. check-in: 5fd8907820 user: [email protected] tags: trunk

Since v0.4.4: - sync: converge clone against writing hub (#17, #18) check-in: a22373f305 user: [email protected] tags: trunk, v0.4.5

A clone against a hub being written to during the session reliably hit "exceeded 100 rounds" once the production scenario exercised the multi-round path. Three compounding root causes:

1. Server-side: emitCloneBatch issued an open-ended `WHERE rid > cursor` query with no upper bound, so blobs committed between rounds extended the queue indefinitely and CloneSeqNoCard{SeqNo:0} never fired. Fixed by capturing max(rid) at the start of the clone session and round- tripping it via CookieCard so the otherwise stateless handler can scope the query: `rid > cursor AND rid <= cap`.

2. Client-side: the libfossil clone client sent CloneCard (clone-mode signal) without CloneSeqNoCard (the actual pagination cursor, per TestHandleClonePagination). Server's cloneSeq stayed 0 every round, masked by every existing clone-client test fitting in DefaultClone- BatchSize=200. Fixed by emitting CloneSeqNoCard on continuation rounds (cycle > 0).

3. Cookie carrier: clone client previously ignored CookieCards on the wire. Now stores and echoes them, mirroring the existing sync client pattern.

Adds three regression tests: - TestCloneAgainstWritingHub (direct issue #17 repro) - TestCloneMultiRoundCursor (catches the cursor bug in isolation) - TestCloneSnapshotBoundExcludesPostSnapshotWrites (snapshot semantic)

The fix is libfossil-internal: protocol carrier is the existing CookieCard with an opaque `lf-clone-cap:<rid>` value, so non-snapshot-aware peers fall back to pre-fix behavior with no wire-format break. check-in: 25e235b387 user: [email protected] tags: trunk

Since v0.4.3: - sync: prevent rid=0 panic on partial xfer (#14, #16) check-in: 5d6bd072ef user: [email protected] tags: trunk, v0.4.4

Auto-version landing now reads this tag at CF build time.

Since v0.4.2: - diff: emit "\\ No newline at end of file" for byte-fidelity with fossil (#10) - repo: expose ResolveVersion + ReadFileAt for symbolic versions (#11) - xfer: always emit project code on push/pull cards (#12) - landing: inject version from latest git tag at build time (#13) check-in: d5dd3dbc3e user: [email protected] tags: trunk, v0.4.3

Replace the three hardcoded "0.4.2" strings in the landing layout with {{ .Site.Params.version }} template directives. Add scripts/sync-version.sh that reads the latest git tag via 'git describe --tags --abbrev=0' and rewrites hugo.yaml's params.version line. Wire it into wrangler.jsonc's build command so Cloudflare's git-integrated build picks up new tags automatically — push a commit to main with a fresh tag, CF rebuilds, the landing shows the new version with no PR required to bump strings.

Footer year switched to {{ now.Year }} so 2027 doesn't surprise anyone.

Local fallback: when no tags are reachable (very early dev), the script exits cleanly and the layout's 'default' filter falls back to the value already in hugo.yaml. check-in: ab1e500bc4 user: [email protected] tags: trunk

The encoder emitted bare 'push\n' / 'pull\n' cards in code paths where the project code wasn't passed explicitly. The upstream Fossil C server's xfer parser requires at least the project code on push and both project + server codes on pull, rejecting 0-arg cards. This broke libfossil <-> fossil interop on the first sync round.

Make the project code a required parameter on the encoder and update call sites to thread it through. The libfossil server-side decoder is tightened to match — it previously accepted 0-arg cards, which masked the encoder bug during Go-to-Go testing. check-in: 4082bffaa5 user: [email protected] tags: trunk

* feat(repo): expose ResolveVersion + ReadFileAt for symbolic versions

Repo.ReadFile took a raw int64 RID, leaving every library caller to re-implement symbolic-version resolution (tip, trunk, branch name, UUID prefix) — work the CLI's resolveRID already does inside the cli package. Move the resolver onto Repo as ResolveVersion, refactor cli/shared.go to call it, and add Repo.ReadFileAt(version, path) so callers can read by name without first calling ResolveVersion themselves.

Adds ErrVersionNotFound for the not-found case and an ambiguity error for prefix collisions.

* chore(cli): drop unused openInternalRepo helper + repo import

Now-dead code: resolveRID was its only caller, and resolveRID is now a one-line wrapper around Repo.ResolveVersion that doesn't need the inner handle. Also removes the now-unused internal/repo import that the helper brought in, and refreshes the cli SDK page accordingly.

* chore(go.work.sum): pin transitive deps from docs toolchain

gomarkdoc pulls these via go install during make docs-gen-sdk; commit the lockfile entries so contributors see no go.work.sum drift after a docs build. check-in: 7eb46a9635 user: [email protected] tags: trunk

Upstream fossil's diff output marks any side whose content does not end with a newline using the standard POSIX marker. internal/diff/diff.go was stripping trailing empty lines via splitLines and never emitting the marker, producing diffs that decoded cleanly Go-side but diverged from the upstream tool — recipients reconstructing file content from the diff would silently introduce a newline that was not in the source.

Track per-side ends-with-newline status and emit the marker after the final hunk line of each side that lacks the terminator. check-in: 2a11bf319a user: [email protected] tags: trunk

Implements all seven recommended changes from the docs audit:

1. SDK pages now show package names in nav. Add scripts/prepend-sdk-frontmatter.sh that prepends '--- title: <pkg> ---' to each gomarkdoc-generated api.md; wired into 'make docs-gen-sdk'. Sidebar previously showed five undifferentiated 'API' entries; now shows cli, db, dst, libfossil, observer/otel.

2. /examples/ populated with paragraphs + GitHub deep-links for embed-repo-api and wasm. No longer a placeholder pointing back at the repo.

3. Architecture page gets an inline SVG mirroring the landing-page diagram. Self-contained CSS (uses currentColor + ember accent) so it tracks Hextra's light/dark theme without depending on the landing's CSS scope.

4. Cross-links added: quickstart -> production + extension-points, migration -> architecture, architecture -> extension-points. Extension Points and Production patterns are now reachable via prose, not just the sidebar.

5. /docs/ landing expanded from 3 sentences to a 7-line subsection map calling out each page's purpose.

6. Version indicator in navbar (libfossil v0.4.2). Override layouts/_partials/navbar-title.html mimics Hextra's default and appends a small mono version tag from .Site.Params.version.

7. New /docs/production/ page covering error handling, context cancellation, telemetry wiring, repo-handle lifetime, and driver selection — the patterns the quickstart's log.Fatal shortcuts gloss over. check-in: f30849a466 user: [email protected] tags: trunk

- README: replace placeholder URL with the live CF Workers URL, add a Docs badge to the badge cluster so the link is prominent on the GitHub repo page. - hugo.yaml: set baseURL so Hugo emits absolute canonical <link> tags and a real sitemap.xml instead of root-relative paths. - SDK pages: refresh gomarkdoc output (drifted from current Go doc comments since the original generation). Same regen that PR #7 contains; either-order merge is a no-op for the second.

The GitHub repo's About sidebar now also links to the docs site (set via 'gh repo edit --homepage'). check-in: 0ca5c54ee6 user: [email protected] tags: trunk

* site(landing): SVG architecture diagram + SDK regen

- Replace §04 ASCII diagram with an SVG mirroring the dagnats style: three columns (callers / core / plugins), ember flow arrows with CALLS / DELEGATE / EVENTS / PERSIST labels, bottom box for .fossil (SQLite). Picks up the layout's existing .diagram CSS with no new styles. - Refresh gomarkdoc-generated SDK pages — they had drifted from current Go doc comments since the original generation.

* docs(site): drop redundant body H1s from frontmatter pages

Each of the four moved docs (architecture, extension-points, migration-from-fossil, testing) and the two hand-written driver _index pages had a body '# Title' that duplicated their frontmatter 'title:'. Hextra renders both, producing the visible doubled headline. Removing the body H1 leaves Hextra's frontmatter-driven page heading as the single source.

SDK api.md pages from gomarkdoc are unaffected — they have no frontmatter, so Hextra promotes their body H1 to the page title without duplication.

* site(landing): bump version refs to v0.4.2 check-in: d1016bd2ef user: [email protected] tags: trunk

* design: docs site + landing page + Cloudflare auto-deploy

Mirror dagnats's docs/landing infrastructure for libfossil: a single-page landing at site/index.html (forked CSS/HTML, content rewritten), a Hugo + Hextra docs site at docs/site/, gomarkdoc-generated SDK reference under docs/site/content/docs/reference/sdk/, llms.txt artifacts, and a wrangler.jsonc that lets Cloudflare's git integration auto-deploy on push to main. Existing docs/*.md files move under docs/site/content/docs/.

* design: correct landing-page path to Hugo layout override

dagnats's working landing page lives at docs/site/layouts/index.html (a Hugo home-page layout override), not at site/index.html. The top-level site/index.html in dagnats is an older draft that the build pipeline ignores. Match the working setup.

* design: align visual system + section list with live dagnats landing

The earlier description came from dagnats's stale top-level site/index.html (JetBrains Mono + Instrument Serif italic + ember/teal accents, 8 sections). The live Hugo layout that actually serves at / uses Geist Mono only, prefers-color-scheme dual-mode (warm cream / warm dark), a single ember accent, and 7 sections (Install, What it is, Why you might want it, How it fits together, Compared to neighbors, What it is not, Get going). Update the spec to match what we are actually mirroring.

* design: drop SDK regen from CF build to match dagnats exactly

Earlier draft had CF install Go + gomarkdoc and re-run docs-gen-sdk on every deploy. dagnats does not do that; it relies on the pre-commit hook to keep committed SDK pages in sync, and ships only Hugo + llms-gen + hugo --minify in the CF build. Match that: smaller, faster CF build, no Go-runtime assumption about Workers Builds, and a true 1:1 mirror.

* plan: docs site + landing + CF auto-deploy implementation steps

11 tasks covering Hugo+Hextra bootstrap, content moves, gomarkdoc SDK generation with pre-commit drift check, llms.txt artifacts, Makefile targets, landing page fork from dagnats's live Hugo home layout, wrangler.jsonc for CF Workers Static Assets, README update, and PR creation.

* docs(site): bootstrap Hugo + Hextra skeleton

* docs(site): move existing docs into Hugo content tree

* docs(site): fix links + H1s broken by docs-tree move

- Drop .md extensions on Hugo cross-doc links - Switch source-code references to absolute github.com/danmestas/libfossil URLs (relative ../ paths broke when files moved 3 dirs deeper) - Align H1 of architecture.md and migration-from-fossil.md with their frontmatter titles

* docs(site): add quickstart, examples, and SDK section stubs

* docs(site): tighten quickstart snippets + reference index

- Make the Sync example self-contained (open + sync, full main, real signatures) - Bridge from Clone to Open with a one-line prose link - Add a sentence to reference/_index.md so it does not render blank - Use the project's canonical "pure-Go, CGo-free" phrasing

* docs(site): add docs-gen-sdk Makefile target + commit generated SDK pages

* docs(site): pin gomarkdoc version + replace empty driver stubs with prose

- Add GOMARKDOC_VERSION Makefile variable so contributors don't drift on output format when a new gomarkdoc ships - Drop the two driver packages (modernc, ncruces) from gomarkdoc and hand-write _index.md pages explaining the register-on-import contract; the stubs were byte-stable but rendered as visibly broken documentation

* docs(site): add gen-llms-txt.sh + docs-gen-llms target

* docs(site): switch llms-full frontmatter strip from sed to awk

The sed range-negation pattern '/^---$/,/^---$/!p' silently re-opens a suppression range on any bare '---' in body content, eating Markdown horizontal rules and embedded YAML in fenced code blocks. The awk pattern strips the frontmatter only when it begins on line 1, leaving body content intact regardless of how many '---' lines it contains.

* hooks: pre-commit fails when SDK pages drift from code

* hooks: pre-commit re-installs gomarkdoc if it's the wrong version

The drift check's existing gate just looks for gomarkdoc on PATH. A contributor with an older gomarkdoc installed locally would have the gate pass and then run regeneration with their stale binary, producing either a silent false-negative or a confusing OUT-OF-DATE error with no version hint. Compare the installed version against GOMARKDOC_VERSION from Makefile and re-install on mismatch. Best-effort; offline contributors can still commit.

* docs(site): add docs-serve, docs-build, docs aggregate targets

* docs(site): make docs aggregate parallel-safe via explicit deps

The docs aggregate previously listed gen-sdk, gen-llms, and build as side-by-side prereqs. Make doesn't guarantee left-to-right ordering under -j, and gen-llms reads what gen-sdk writes, while build reads both. Encode the chain explicitly so 'make -j docs' is correct. Side benefit: 'make docs-build' standalone now produces a complete deployable site, matching contributor expectation.

* site(landing): fork dagnats home layout; rebrand chrome

* site(landing): rewrite §01–§07 content for libfossil

* site(landing): use .ascii class for diagram + fix §03 phrasing

* deploy: add wrangler.jsonc for CF Workers Static Assets

* docs: point README at the new docs site check-in: e2a2f28861 user: [email protected] tags: trunk

* sync: defer manifest crosslink until referenced blobs arrive

Repo.HandleSync (added crosslink in v0.4.1 PR #4 fix 3) crosslinked each manifest as soon as its blob arrived, even when the manifest referenced blobs that hadn't yet been received in the same session. A receiver that Pulled during a multi-blob delivery saw the manifest in event/leaf/mlink while one or more file blobs were still in flight, and a downstream Checkout.Update walking those manifests via manifest.ListFiles hit 'expandUUID: blob not found for uuid <hex>'.

Surfaced by agent-infra's hub-and-leaf scaling trial under 16-way concurrent fork+merge cycles (agent-infra trial #10, see docs/trials/2026-04-25/trial-report.md). The 3-leaf friendly case in agent-infra's hub-leaf-e2e doesn't exhibit it because the broadcast cadence is slower than the multi-blob delivery window.

Fix: manifest.Crosslink now defers a Checkin candidate when any referenced blob (F-card files or B-card baseline for delta manifests) is not present locally. The deferral is implicit: the candidate query selects rids without an event/tagxref/forumpost/attachment row, so a deferred manifest re-appears in the next sweep automatically. Since HandleSync runs Crosslink whenever filesRecvd > 0, the next sync round that delivers the missing blob also re-runs the sweep and links the manifest fully.

Divergence from fossil-scm/c: fossil's reference handles this via 'rcvfrom' tracking and ipctest-style deferred crosslink flushes; the Go port reuses the existing whole-repo sweep semantics by checking blob presence at sweep time. Documented inline in manifest.Crosslink and missingCheckinRefs.

Test coverage: - TestCrosslink_DefersCheckinWithMissingFileBlob — manifest with missing F-card blob is deferred (no event/leaf/mlink rows), then fully linked once the file blob arrives. - TestCrosslink_DefersDeltaCheckinWithMissingBaseline — delta manifest with missing B-card baseline is deferred; baseline + delta both link in the next sweep once the baseline blob arrives. - TestCrosslink_LinksWhenAllBlobsPresent — confirms the unchanged synchronous path (all blobs present before Crosslink runs).

All three tests fail without the fix (verified by reverting the crosslink.go change locally) and pass with it.

* manifest: assert non-nil repo and non-empty uuid in blobPresent

Tiger Style: panic at the boundary on programmer errors. Calling blobPresent with a nil receiver or empty uuid is always a bug — empty uuid would silently return false (no row matches the empty-string predicate), masking the upstream caller that lost track of which uuid it intended to check.

The peer helpers Crosslink and missingCheckinRefs already panic on nil inputs; blobPresent was the gap.

* manifest: order Crosslink candidate sweep by rid

Without an explicit ORDER BY, SQLite is free to return candidate rows in any order convenient to its query planner. The deferred-crosslink sweep re-discovers the same rids across multiple sync rounds, so any implicit ordering would let blob-arrival timing leak into observable behavior: per-defer slog streams, pending-item processing order, and any future caller that consumes Crosslink's side effects in sequence.

ORDER BY b.rid pins the sweep to insertion order — the same order blobs were stored — which is the natural arrival order for a single sync and a stable canonical order across replays.

* manifest: emit deferred rids and missing-blob list in Crosslink rollup

The session-end Info log previously reported only counts. When a sync ends with deferred manifests, an operator needs to know which manifests are stuck and which UUIDs they are waiting on — otherwise a permanently-stuck deferral (blob never arrives) is invisible until someone diagnoses it from the per-defer Debug stream.

Add deferred_rids and missing_blobs fields. Sort missing_blobs so the rollup is byte-identical across runs that defer the same set, regardless of map iteration order.

* manifest: cover P-card non-deferral and shared-blob defer cases

The original three defer tests covered F-card and B-card defer plus the all-present happy path. Three gaps remained:

* Multi-manifest sharing a missing blob: prove both manifests defer together and both link when the shared blob lands. Without this, a regression that links one but not the other would slip past.

* Missing-parent (P-card) intentionally does NOT defer: the deferral check is scoped to F-cards and B-cards because P-card targets do not break Checkout.Update. Deferring on missing parents would deadlock partial clones with pruned ancestors. Pin the contract with a regression test.

* F-card duplicate-UUID dedup: the seen-map in missingCheckinRefs already deduplicates, but the behaviour was untested. Two F-cards with different filenames pointing at the same blob must defer once and link to two mlink rows on arrival.

* xfer: handle empty ServerCode/ProjectCode without arg-count error

Repo.Sync(ctx, transport, SyncOpts{Push: true, Pull: true}) with both codes empty previously wrote "push \n" to the wire. The decoder's strings.Fields collapsed adjacent spaces and failed with "xfer: pull requires 1-2 args, got 0".

Encoder now skips trailing empty args. Decoder accepts the verb with 0 args as "no specific code requested" — the same semantic as the existing 1- and 2-arg forms after fossil's normal arg interpretation.

Also fixes session.cardSummary's slice-out-of-range panic on empty codes by clamping the prefix length.

Adds a roundtrip test for the empty-codes path.

* sync: continue round-loop while either side has pending work

Sync's round-loop previously exited when the user passed Pull=false even if the peer was still requesting gimmes. The user's Pull flag controls whether we *request* new content; it should not gate whether we *service* the peer's gimmes. Symptom: agent-infra's Manager.Push had to pass Pull=true as a workaround to drive multi-round transfers.

Root cause: HandleSync.handleIGot gated gimme generation on pullOK alone. Push-only clients send igot cards from sendUnclustered every round (they always announce what they have), but the server would silently drop them because pullOK was false — leaving the server holding only what the client pushed proactively.

Continue the loop while any of: pending ungots to push, pending pulls to request, gimmes to service. Exit only when a round completes with no work in either direction. Concretely the gate is now (pushOK || pullOK), restoring parity with fossil-scm's client_sync continuation logic.

Adds an integration test (TestPushOnly_DrainsServerGimmes) that exercises the push-only path end-to-end and a unit test that asserts gimme emission under push-only handleIGot.

* sync: crosslink incoming manifests so hub-side state stays consistent

Repo.HandleSync previously stored manifest blobs in 'blob' but did not populate the relational tables (event, leaf, plink, mlink) that the timeline, fork detection, and clone protocol depend on. As a result:

- Leaves pulling from a hub got blobs but stale 'leaf' state, so WouldFork reported phantom siblings. - Verifier-clone of a hub repo failed with "sync.Clone: exceeded 100 rounds" because 'mlink' was empty and the clone protocol couldn't traverse manifest parents. - Hub timelines were always empty even after successful Push.

HandleSync now invokes manifest.Crosslink (the same scanner Clone and Sync run after convergence) when at least one file landed in the round. The scanner walks blobs that have no event/tagxref/ forumpost/attachment row yet, parses them, and populates the relational tables. Local commits already produced this state via manifest.Checkin's existing crosslink path; incoming syncs now match.

Coverage spans every artifact type the existing scanner already supports (ci, wiki, ticket, event, attachment, cluster, forum, control). The agent-infra trial that surfaced this only exercises 'ci' artifacts, so the divergence note in the prompt described the narrowest possible repair; reusing the existing scanner avoids the divergence and matches the C reference's full coverage.

Adds two tests: TestHandleSync_CrosslinksReceivedManifest verifies event/leaf/mlink rows after pushing a single checkin; the leaf- transition test verifies the parent leaves the leaf table when the child lands and that plink connects them. check-in: 3f290b159f user: [email protected] tags: trunk, v0.4.1

Expose a public wrapper around the internal blobAt helper so callers can read a file's content at a specific checkin rev without going through a checkout. Returns ErrFileNotFound (wrapped) when the path is not tracked in the given checkin.

Diff already used blobAt internally; this just surfaces the primitive. check-in: d8b2b1bca4 user: [email protected] tags: trunk, v0.3.0

Adds Repo.Merge(srcBranch, dstBranch, message, user) which resolves each branch tip, finds the common ancestor, runs ThreeWayText.Merge against the union of source and destination filesets, and — if every file merges cleanly — records a merge commit whose primary parent is the dst tip and whose secondary parent is the src tip (plink isprim=1 / 0).

Any unresolved conflict aborts the whole operation with a *MergeConflictError listing the affected files; callers can check for it via errors.Is(err, ErrMergeConflict). Agreed deletions are honored, and one-sided modify/delete conflicts are reported as conflicts rather than silently dropped.

Supporting changes:

- CheckinOpts.MergeParents plumbed through manifest.Checkin so the P-card and plink rows carry every parent (previously the fresh-commit path only tracked one parent, unlike the sync path in crosslink.go). - manifest.Checkin now calls tag.PropagateAll(primaryParent) after inline T-card processing, mirroring crosslink.applyInlineTags. Without this, a child commit on an existing branch would not inherit its parent's branch tag, so BranchTip would return stale tips for any non-initial commit. - Repo.BranchTip(name) resolves a branch to its latest checkin via tagxref; secondary ORDER BY rid keeps the lookup deterministic when multiple commits share a julian-day bucket. - CommitOpts.MergeParents exposed on the public API so arbitrary callers can author merge commits without going through Repo.Merge.

Tests cover clean merges of divergent edits, conflicting edits, one-sided additions, missing branches, same-branch input, multi-parent plink verification, and BranchTip resolution on both trunk and named branches. Green on both the modernc and ncruces drivers plus the OTel observer. check-in: 461f168708 user: [email protected] tags: trunk, v0.2.0

Fills in the TODO stub in repo_history.go so Repo.Diff returns a real unified diff for a file between two checkins instead of a "not yet implemented" error.

The implementation piggybacks on internal/diff.Unified, which already produces Myers-algorithm output. A small blobAt helper walks the manifest at each checkin, resolves the file's UUID to a blob RID, and expands the content. An absent file on a side is treated as empty bytes so pure additions and deletions render as clean insert/delete hunks. Identical content on both sides returns an empty slice rather than a DiffEntry with an empty Unified string, which keeps callers from having to distinguish "no diff" from "missing output".

Adds repo_history_test.go covering modification, identical content, addition, deletion, absent-both-sides, and the empty-filePath error. check-in: a3fbae135e user: [email protected] tags: trunk

Replace blocks in published go.mod files are ignored by consumers but cause `go install @v0.1.0` to fail module resolution. Local development continues to work via go.work, which substitutes workspace modules without requiring go.mod replace entries.

Also add observer/otel to the workspace for consistent local builds. check-in: 71751cf461 user: [email protected] tags: trunk, db/driver/modernc/v0.1.0

libfossil is a pure-Go implementation of the Fossil SCM, reading and writing the same .fossil SQLite repository format as upstream Fossil.

This is the first public release of code that previously lived in a private repository. The module path is github.com/danmestas/libfossil and the binary is named libfossil to avoid $PATH collisions with upstream fossil.

Included: - Repo lifecycle (create, clone), checkout/checkin, timeline, merge, diff, annotate, shun, tag, branch, bisect, stash, undo/redo - Sync-over-HTTP protocol (client and handler) - Two SQLite drivers: modernc (pure Go, default) and ncruces (wasm-capable) - Observer interfaces (SyncObserver, CheckoutObserver) with an optional OpenTelemetry implementation in observer/otel (separate submodule) - Deterministic simulation testing harness with BUGGIFY fault injection - Kong-based embeddable CLI under cli/ and a standalone binary under cmd/libfossil/ check-in: f5744359d3 user: [email protected] tags: trunk