Timeline

delta.Apply's one-time capacity upgrade (#127) only engages above the 64 KiB threshold, so it is a no-op for the typical source file under 64 KiB. But below the threshold Apply already sizes its output buffer exactly, so there was never a 5x overhead there to remove -- extending the threshold buys nothing. The real per-link cost of a chain walk was elsewhere, and a profile of a plain Expand of a deep sub-64 KiB chain shows it: a fresh ~32 KiB zlib history window allocated per delta load (~27% of bytes), one full-size Apply output buffer per link (~16%), and a fresh inflate buffer per link. All three are O(depth) allocations for a single cold Expand, which is O(depth), making a full-history walk O(n^2).

Attack the per-link constant inside the one choke point every expansion goes through, content.expandChain, leaving the on-disk chain shape untouched:

- delta.ApplyInto(dst, source, delta) reconstructs into a caller-supplied buffer, reusing its capacity. Apply is now ApplyInto(nil, ...), unchanged. A reused buffer's capacity is proven prior work, not a wire-header claim, so it is a safe starting size; the anti-DoS growth path (growOutputCap, keyed off len(delta) not the targetLen header) is preserved for buffers a target outgrows. - blob.Inflater loads a run of blobs through one zlib reader, reset per blob instead of rebuilt, paying the ~32 KiB window setup once per chain. Each Load still returns a freshly owned buffer, so no returned-slice-aliasing contract leaks to callers. - expandChain's plain-Expand replay (put == nil) ping-pongs two output buffers -- each link's target reuses the buffer from two links back, whose content was already consumed as the previous link's source, so dst and source never alias. The caching path (put != nil) is unchanged: put takes ownership of every materialized buffer, so those cannot be reused underneath it, but it still shares the reused zlib reader for its delta loads.

Measured on Apple M4 Pro, BenchmarkExpandOldestVersion, median of 3, 40x:

sub-64 KiB (~26 KB file), Expand of the oldest revision: depth 11: 883 KB -> 232 KB/op (3.8x), 1.12 ms -> 0.90 ms depth 49: 3.57 MB -> 341 KB/op (10.5x), 4.55 ms -> 2.03 ms depth 199: 14.2 MB -> 772 KB/op (18.4x), 14.84 ms -> 7.25 ms

>64 KiB (~206 KB file), no regression -- ping-pong + reader reuse help it too: depth 11: 4.11 MB -> 1.27 MB/op (3.2x), 3.40 ms -> 2.74 ms depth 49: 16.5 MB -> 1.81 MB/op (9.1x), 13.02 ms -> 7.69 ms depth 199: 65.2 MB -> 3.92 MB/op (16.6x), 46.3 ms -> 26.6 ms

Sub-64 KiB allocation is now near-flat in depth instead of linear in depth*targetLen. expand_reuse_test.go guards the reuse; delta and blob gain allocation-reduction unit tests. check-in: a2bcca2535 user: [email protected] tags: trunk

A stock fossil 2.28 client cloning a go-libfossil server transferred ~2.06x the repository's stored size, while the server's own response bodies measure 1.04x and a fossil-serving-fossil control measures 1.03x. The gap was a full second content-transfer pass, not a card-encoding problem.

Root cause is card order. Canonical's client re-issues `clone 3 SEQNO` every round it sees a server `push` card while its clone cursor is still positive (xfer.c:2706), and it learns the cursor reached zero only from the clone_seqno card. Canonical's server therefore emits the batch, then clone_seqno (xfer.c:1571), then push (xfer.c:1577). go-libfossil emitted push *first*, so the client queued one more clone request before it processed the terminal clone_seqno 0 — and the server served the entire repository a second time.

Relocate the clone-mode push-card emission to after processDataCards so it trails the batch's clone_seqno card, matching canonical's order. The clone now completes in a single content pass: re-measured end-to-end at 1.03x stored, identical to the fossil control (1.00x ratio). check-in: 1fd6c06a65 user: [email protected] tags: trunk

docs/site was the one module in the tree with no CI compile gate. It has its own go.mod but zero Go source of its own -- the module exists only to import the hextra theme via Hugo Modules -- so `go build ./...` there matches no packages and gates nothing (and `go vet ./...` fails on "no packages to vet"). The equivalent compile gate for a Hugo module is a Hugo build: it resolves the module import, compiles templates, and renders content, failing loud on a broken import or template.

Add a test-docs job that installs pinned Hugo Extended and runs `hugo --gc --minify` in docs/site, matching how the driver/observer submodules each get their own fail-loud step. check-in: be89e6397b user: [email protected] tags: trunk

* fix: emit renames as a single F-card with the "w" placeholder (#51)

buildCommitFiles seeded its working set from the parent manifest and layered vfile changes on top, but never consulted vfile.origname. A pure rename has identical content, so ScanChanges clears its chnged flag; the new name then never entered the working set while the old name carried forward from the parent, so the manifest emitted the file's content under its old pathname and dropped the rename entirely.

Drive renames off vfile.origname, mirroring canonical Fossil's manifest query (src/checkin.c:1939): applyRenames retires the old pathname and records it on the surviving new-name entry, so a rename is emitted exactly once. The unchanged-content pass then loads the blob and preserves OldName.

Serialize the rename card to match stock Fossil: force a "w" permission placeholder when a renamed file's perm would otherwise be empty, keeping the prior-name field in its 4th positional slot (src/checkin.c:1999). Thread OldName through manifest.File and storeFileBlobs so the commit path can carry it.

Verified the exact card "F greet.txt <uuid> w hello.txt" against a real fossil binary for the same rename scenario.

* fix: never emit a renamed-then-deleted file in the commit manifest

applyRenames enumerated every vfile row with origname != pathname, including rows marked deleted, and unconditionally seeded a new-name entry for them. A file that was renamed and then unmanaged therefore committed as a rename F-card under its new name. The deleted-files pass could not catch it: it only knows the file's current pathname, so deleting the new name from a map keyed on parent-manifest pathnames was a no-op and nothing retired the old name.

Canonical Fossil excludes deleted rows from its manifest query outright (WHERE NOT deleted OR NOT is_selected). Match that: a deleted renamed row now retires both names and emits nothing, keeping the existing guard that leaves the old name alone when a live vfile row has reclaimed it.

Also move the deck marshaller's prior-name field inside the UUID guard. The field is positional and only meaningful after a hash; a deletion card carrying one would serialize as "F name oldname", with the old name misread as the UUID.

Tests: rename-then-delete emits no card under either name; a rename chain that recycles a freed pathname emits both cards with the right prior names and content. check-in: 29c2c5c6f5 user: [email protected] tags: trunk

* content: add delta-chain depth benchmark for #85 re-measurement

BenchmarkExpandOldestVersion expands the oldest version of a ~26 KB, 400-line file at 12/50/200 revisions, mirroring the table in #85. The chain is built through content.Deltify so the on-disk shape matches what content_deltify produces: newest version whole, each older version a delta against the one newer.

Reports ns/op plus B/op and allocs/op so the per-link allocation profile the issue is about is directly observable.

* content: straddle delta.Apply's 64 KiB cap in the chain-depth benchmark

BenchmarkExpandOldestVersion measured a ~26 KB file, which sits below delta.Apply's deltaInitialCap: for targets that small the output buffer is allocated at exactly targetLen and the capped/growOutputCap path is never entered, so the arm cannot observe #81/#82 at all.

Parameterise the chain builder on file size and add a ~206 KB arm above the cap, so the two arms isolate the threshold rather than leaving it inferred from reading delta.go. Also move the chain-depth metric after the loop -- ResetTimer deletes user-reported metrics, so the depth reported before it never reached the output.

Refs #85

---------

Co-authored-by: Dan Mestas <[email protected]> check-in: c3338c4d38 user: [email protected] tags: trunk

The clone command hardcoded a 10-minute context deadline with no way to raise or disable it, and the deadline did not reliably interrupt a slow clone.

- RepoCloneCmd gains a --timeout flag (default 10m, matching the old behavior; 0 disables the deadline for very large repositories). cloneContext branches away from context.WithTimeout for a non-positive timeout -- WithTimeout(parent, 0) yields an already-expired deadline, which would abort instantly the very clone the flag exists to unbound.

- The whole-repository crosslink sweep now observes context cancellation. It runs after the exchange-round loop, so it had no round boundary of its own at which the deadline could fire -- on a large repository it could run long past the deadline uninterrupted. manifest.CrosslinkContext checks the context every 256 candidates; manifest.Crosslink stays a background-context wrapper so its ~30 existing callers are unchanged. processResponse gains the same batched check so a large single-round batch is interruptible too, and the sync session's and the serve handler's post-convergence sweeps are wired to their contexts for the same reason.

Each cancellation point is pinned by a test that fails without it: the crosslink and processResponse tests drive a context that survives the first poll and dies before the second, so only the batched check can observe it; the crosslink-sweep test cancels after a zero-card final round, so nothing before the sweep is left to notice.

Also adds a clone -> serve -> reclone fidelity regression guard (#114), built with the canonical fossil binary rather than this library's Commit. Commit does not deltify, so a corpus built through it holds zero delta rows and never reaches the serve side's deltified-row expansion branch -- a green result from such a corpus proves nothing about #114. The guard now asserts the source holds delta rows before measuring anything, and compares the second-generation clone's tip, artifact count, blob-store size, delta-row count and checked-out tree content against the source.

Measured against that corpus (71 artifacts, 54 delta rows), #114 does not reproduce: gen1 and gen2 both hold the same 54 delta rows, the same tip and artifact count, a 1.009 store ratio, identical checked-out content, and pass fossil test-integrity. The receive path re-deltifies what the serve side expanded, via content.Deltify on the crosslink mlink path, so the expansion the serve side performs does not survive into the store. check-in: a02c9572da user: [email protected] tags: trunk

Adds a wire-efficiency regression guard the issue flagged as missing: correctness-only round-trip tests never asserted card type or wire cost, so the file-card-instead-of-cfile amplification #98 fixed could silently return. TestCloneServesCompressedNotExpandedContent serves a small compressible corpus, clones it over the real HTTP transport, and fails if the server emits any expanded file card or if served bytes exceed 3x the repository's stored size. Card type is the load-bearing signal: the reply container is compressed, so a raw-file-card regression on compressible content hides inside a byte ratio -- the 3x bound is only a coarse backstop.

Also adds a WIRE_MEASURE-gated manual harness that re-measures serve-side amplification against a real fossil client through a byte-counting proxy, with a fossil-serving-fossil control. Current main: 2.06x stored end to end (server response bodies alone are 1.04x, matching the fossil control's 1.03x), down from the 17.9x this issue reported. check-in: 1a73119ed4 user: [email protected] tags: trunk

FossilEncode/FossilDecode implemented only 3 of the 8 escapes the artifact/sync format §4.3 mandates (\s, \n, \\). A field value carrying a real TAB, CR, VT, FF or NUL byte -- as canonical fossil writes it -- was mis-decoded and no longer re-marshalled to its source wire bytes, changing the artifact hash. Unrecognized escapes were also left literal instead of dropping the backslash per spec.

Add the remaining escapes (\t \r \v \f \0) to both directions and make an unrecognized escape drop its backslash, matching canonical fossil fossilize/defossilize (verified against fossil 2.28). Every caller -- A/C/H/L/J/F/T/U cards and the xfer cards -- shares these functions and gets the fix.

Closes #119 check-in: 1bfa6b2094 user: [email protected] tags: trunk

The test step runs `go test $(go list ./... | grep -v '/dst')`. When a package fails to compile, `go list` prints an error to stderr and omits that package from stdout, so it silently drops out of the test run rather than failing CI. `go build ./cmd/libfossil/` only covered the CLI binary's dependency closure, which missed internal/deck when it broke.

Add `go build ./...` ahead of the test step on both driver-tag legs (modernc and test_ncruces) so any package that fails to compile is a loud, fail-fast build error. This subsumes the narrower cmd/libfossil build, so that step is removed rather than kept alongside. check-in: fa558f254d user: [email protected] tags: trunk

The xfer HTTP server let net/http frame large clone/sync replies as Transfer-Encoding: chunked with no Content-Length. Release-tagged fossil <=2.23 reads the reply length only from Content-Length (src/http.c leaves iLength negative otherwise) and aborts with "server did not reply" on any chunked reply above ~256KB, so those clients could not clone a repository larger than that from a libfossil server.

writeXferResponse already materialises the whole response via Encode before the first write, so the exact body length is known for free. Set Content-Length explicitly; net/http then never falls back to chunked.

The fix is unconditional rather than negotiated. The triage proposed sniffing a client capability and keeping chunked for capable clients to avoid the buffering cost of Content-Length. That premise does not hold here: the server never streams -- Encode always returns a complete slice -- so chunked saves no memory over Content-Length, and #88's per-clone bound (DefaultCloneBatchBytes) still caps each reply either way. Content-Length is exactly what canonical fossil's own server emits, so it is strictly the more compatible framing for every client, including builds whose version banner cannot be trusted (#86).

Tests: a reconstructed <=2.23 client that requires a non-negative Content-Length and rejects chunked framing on a >256KB clone response (the discriminating test, since real fossil 2.28 tolerates chunked and cannot reproduce the failure), plus a modern-client no-regression clone of the same corpus. The existing real-fossil multi-batch clone interop test continues to pass unchanged.

Fixes #101 check-in: b3139353e1 user: [email protected] tags: trunk

* Wire manifest.AfterDephantomize into the phantom fill path

AfterDephantomize was a faithful, tested port of Fossil's after_dephantomize (content.c:389-456) that nothing in production ever called — it crosslinked correctly in isolation but sat disconnected from the real receive path.

storeResolvedContent fills a phantom row in place when the full content for a previously-unknown UUID arrives (matching canonical's content_put_ex filling a phantom at content.c:626), but stopped short of crosslinking it or any delta children waiting on it. Return the filled rid from storeResolvedContent and call AfterDephantomize on it from storeReceivedFile once the fill transaction has committed — outside the transaction, since AfterDephantomize opens its own connection via repo.Repo.DB() and calling it from inside the tx's closure would contend with the still-open write lock.

Added an end-to-end regression test that delivers a checkin manifest as a delta against a base that hasn't arrived yet, then delivers the base, and asserts the manifest is crosslinked (its event row exists) immediately — with no explicit call to manifest.Crosslink anywhere in the test. This proves AfterDephantomize fires on the real fill path, not only under its own unit tests.

* Log crosslink failures during AfterDephantomize instead of swallowing them

A crosslink error while filling a phantom blob (or one of its orphan/delta dependents) previously vanished silently, leaving storeReceivedFile report success with the artifact filled but un-crosslinked. Surface it via slog.Warn so real corruption on the fill path is at least visible, matching the non-blocking nit from review. check-in: 94919ca357 user: [email protected] tags: trunk

* fix: select xfer framing by Content-Type (#106); bound clone rounds by the client's decode limit (#109)

seeing which parsed, which made a decompression failure indistinguishable from "wrong framing" (the root of #104). Select framing from the §4 Content-Type instead: application/x-fossil is the §4.1 compressed container, every other type (including application/x-fossil-uncompressed, used by clone-v3 replies) is plain card text — the same rule canonical fossil applies. Verified against fossil 2.28: pull replies arrive as the prefixed container, clone replies as plain text, and no reply is ever an unprefixed raw zlib stream. The former unprefixed-zlib framing, checked first on the unverified belief that fossil emits it, was removed along with the length-prefix/zlib-header aliasing guard that only existed to disambiguate the two compressed framings trial-dispatch conflated. A compressed body that fails to inflate is now reported as the fault it is, never re-fed to the card parser. xfer.Decode takes the Content-Type; byte-oriented callers with no header pass the compressed type explicitly.

crossing one whole, so a round carried budget + one whole artifact. When sub-budget filler preceded a large artifact, that sum exceeded what the client could decode (xfer.MaxDecompressedBytes) and the clone failed with a spurious "declared container length exceeds" error whose threshold depended on incidental filler size. The server now flushes the round and sends an over-bound artifact alone when adding it would cross the client's decode bound, so clonability is a property of the artifact alone. The 2*budget compile-time guard now certifies what it appears to.

* fix: reject unrecognized xfer Content-Type with 415 (#106)

A POST whose Content-Type was absent or unrecognized fell through to the plaintext card decoder, which is exactly the #104 failure shape: possibly- compressed binary fed to the card parser. Request bodies come from untrusted peers, so a body carrying no framing signal is rejected rather than guessed at. Real fossil always sets the header; no conforming peer is affected.

xfer.KnownContentType shares the media-type parsing with the framing check, so the server and Decode agree on what a type means.

Also runs the #109 filler subtests in parallel — serially they pushed internal/sync past the 120s per-package timeout CI uses. check-in: f1236bd031 user: [email protected] tags: trunk

Issue #60's triage was stale: the live truncation bug it describes was already closed on main by 11967ff (#62), which added millisecond rounding to timeToJulian. What remained was a coverage gap, not a defect.

An earlier attempt on this branch also normalized the float64 branch onto the millisecond grid, on the theory that a driver-independent helper is strictly better. That was wrong, and this commit does not do it. A scanned float64 IS the value stored in the row, so snapping it can only move it away from what the row holds. Timeline's pagination cursor depends on the opposite property: its predicate `e.mtime < ? OR (e.mtime = ? AND b.rid < ?)` requires the cursor to compare exactly equal to the row it came from. SQLite's own julianday() computes iJD/86400000.0, a different expression from TimeToJulian's julianEpoch + ms/86400000.0, and the two land one ulp apart on a real fraction of instants — so normalizing alters mtimes written by upstream Fossil, the library's primary real-world input. Raised, the cursor row repeats as the next page's first row; lowered, rows are skipped. Both are silent wrong answers.

The float64 branch therefore stays verbatim, and its doc comment now records why that exactness is load-bearing rather than incidental. The same reasoning applies to tag propagation's `tagxref.mtime < ?`, where SQLite compares the bound value against raw stored floats: an exact bound is the correct one there too, so no normalized variant is introduced for it either.

The one genuine transform stays where it belongs, in the time.Time branch, which must round because ncruces hands back DATETIME columns carrying its own sub-millisecond conversion noise. Its comment now also records the limit of that recovery: an mtime written at finer than millisecond resolution cannot be reproduced through a time.Time at all, and rounding the float64 branch to match would discard precision on both drivers rather than recover any.

Non-functional cleanup: millisPerDay and julianDayFromMillis replace two open-coded copies of the same expression.

Coverage added (no build tag; these run under both drivers): - db: TestScanJulianDayFloatBranchIsBitExact sweeps julianday()-computed mtimes and asserts each scans back bit-for-bit, with a guard that fails the test if the sweep ever stops containing divergent values. Verified to fail against the normalizing version and pass against this one. - db: TestScanJulianDayIsDriverIndependent proves the float64 and time.Time representations of one millisecond-aligned write converge, with noise in both directions. TestScanJulianDaySentinelZero pins the "no mtime" sentinel that bisect's `mtime != 0` guard depends on. - tag: TestPropagateAllOrderingAdjacentMillisecond drives the `tagxref.mtime < ?` decision through ScanJulianDay with adjacent-millisecond mtimes; it fails under a truncating conversion and passes under the rounding one, and passes under modernc regardless — the reason this class went unnoticed twice.

Consumer classification (ScanJulianDay): - manifest/timeline.go NewCursor: ordering/equality, requires exactness - tag/propagate.go PropagateAll -> `tagxref.mtime < ?`: ordering (now covered) - tag/propagate.go childMtime -> mtimeQueue.Less: ordering, traversal-order only (visited set makes it result-neutral) - manifest/log.go, bisect/bisect.go: display-only

Known limitation, left alone deliberately: julianToTime truncates where timeToJulian rounds, so the two are not inverses at sub-millisecond resolution. Pre-existing, now documented; ScanTime's consumers are display paths with no ordering boundary. This commit does not widen the gap. check-in: 78a793dd83 user: [email protected] tags: trunk

* fix: return error when clone stalls with phantoms outstanding

Clone's convergence loop broke out of its round loop as soon as the phantom count stopped decreasing and returned success, even when the phantom table was still non-empty. That leaves the caller holding a repository missing content with no error and exit code 0.

Stopping on a stall is still correct — spinning forever would be worse — but reporting it as success is not. checkStop now returns a PhantomStallError when the loop halts with phantoms remaining, giving the caller a programmatically-reachable count plus a sample of the missing UUIDs for diagnosis. A clean clone still returns nil with an empty phantom table.

Fixes #64.

* fix: preserve partial CloneResult when Clone returns an error

Clone's named result return was never assigned on the error paths — cs.run's cloneResult stayed in a local variable, so a caller catching a PhantomStallError (or a crosslink failure) got a nil result even though a partial one, with the round/blob counts collected so far, was sitting right there. Assign it before returning on both paths.

* fix: make phantom stall error sample deterministic

newPhantomStallError previously took an arbitrary 10 entries from map iteration order before sorting, so the sample was nondeterministic across runs when more than 10 phantoms were outstanding. Sort all UUIDs first, then take the lexicographically-smallest 10, matching what the doc comment already implied.

Addresses a non-blocking nit from review on #64. check-in: e5be6a9199 user: [email protected] tags: trunk

* fix: zero-length file commit panic and stale missing-file carry-forward

Two related bugs in the checkin path:

#68 - Committing a zero-length file panicked in blob.Store, which treated an empty artifact (a normal, well-known Fossil blob) as invalid input. Worse, manifest.Checkin's postcondition defer fired unconditionally during the panic's stack unwind and re-panicked with its own unrelated "manifestRid must be positive" message, masking the real cause. Fixed both: blob.Store now accepts zero-length (and nil) content, and Checkin's defer re-panics with the original panic value instead of asserting a postcondition that was never going to hold mid-unwind. Fixing the panic surfaced a second bug in the same family: buildCommitFiles used len(Content) > 0 to detect "already loaded from disk," which can't distinguish a genuinely empty file from one not yet loaded, so a newly added empty file failed with "no RID for unchanged file." Replaced the length sentinel with an explicit loaded-set.

#79 - A tracked file deleted from disk without Unmanage/Remove was silently carried forward into the new manifest with its last-committed content, rather than being detected. Verified against fossil 2.28: canonical fails a commit when a missing tracked file is in scope (either the implicit "commit everything" case, or explicitly named), but leaves an out-of-scope missing file alone. buildCommitFiles now stats every in-scope tracked file before treating it as unchanged and errors clearly, naming the file, when it's gone; a file outside the commit's scope (relevant only to an explicit Enqueue) is left untouched exactly as before. Updated the two existing tests that had documented the old silent-carry-forward behavior as deliberately deferred to this issue.

* fix: add CHANGELOG entry and unwrap-safe stat checks for #68/#79

Adds the CHANGELOG entries the #79 acceptance criteria required for a user-visible behavior change (missing tracked file now hard-fails a commit instead of silently carrying stale content forward) and for the #68 zero-length-file panic fix.

Also switches checkin.go's two os.IsNotExist(err) checks to errors.Is(err, os.ErrNotExist), which correctly unwraps the %w-wrapped os.ErrNotExist that MemStorage.Stat returns. Production was already correct (OSStorage.Stat returns a raw *PathError), but the tests were only exercising a generic fallback error branch instead of the intended "tracked file missing from disk" message. check-in: 0351167ae5 user: [email protected] tags: trunk

* fix: clone emits cfile with stored content per sec7.2, not expanded file

Our clone server was building xfer.FileCard from cache.Expand(...) -- every artifact fully decompressed, and every delta materialized into full content -- where sec7.2 requires cfile cards carrying the repository's stored compressed content retransmitted verbatim. On real repositories this ran 80-220x more bytes on the wire than the protocol intends, and was the real blocker behind #88's clone-batching fix being unable to bound repositories much past libfossil's own size.

emitCloneBatch now reads blob.Load's stored bytes directly (full content for a whole artifact, delta bytes against DeltaSource for a deltified one) and emits xfer.CFileCard instead of xfer.FileCard, with DeltaSrc and USize (the full reconstructed size, from blob.size) populated correctly for delta rows. content.DeltaSource is exported so this and other callers outside internal/content can ask whether a rid is stored as a delta without duplicating the query.

No client changes were needed: session.go already routes CFileCard through the same handleFileCard path as FileCard, and storeDeltaContent already stores a received delta exactly as given via a create-phantom-if-missing lookup on its source, resolving delta-before-base arrival the way #53's phantom storage model was built for.

DefaultCloneBatchBytes is left at its current value rather than retuned toward canonical's 5,000,000 -- that constant's tuning is #109's territory now that bytesSent tracks stored bytes instead of expanded ones, and retuning it here risks colliding with that work.

* fix: expand deltified rows during clone instead of sending delta cfile

Code review reproduced a real interop regression: emitCloneBatch sends cards in ascending-rid order, but content.Deltify always deltifies the older artifact against the newer one, so a delta's DeltaSource almost always names a rid greater than the delta card's own rid. That means a delta cfile routinely reaches the client before the very source card it names. libfossil's own client tolerates the forward reference (storeDeltaContent phantoms the unseen source and resolves it later), but real fossil 2.28 does not -- the source stays a genuine, unfilled phantom, and content_get() on it during the client's post-clone rebuild pass returns without initializing its output blob, tripping an assertion in blob_copy (blob.c:397) and aborting the client mid-rebuild. Verified independently against a real fossil 2.28 binary cloning a libfossil server holding a delta chain.

emitCloneBatch now expands a deltified row to full content before sending it as a plain (non-delta) cfile, rather than forwarding the stored delta bytes with DeltaSrc set. The card is still zlib-compressed, so the wire-size win over the old FileCard/Expand path survives for every row; only the additional delta-retransmission savings are given up, and only for deltified rows. Re-enabling that needs either a source-before-dependent send order or confirmed forward-reference tolerance in canonical, and is left as a follow-up rather than blocking this fix.

TestEmitCloneBatchExpandsDeltifiedRows (renamed from TestEmitCloneBatchRetransmitsDeltaAsDelta) now pins the new contract: no DeltaSrc, full-sized payload. TestCloneRealFossilWithDeltaChain adds an end-to-end check against a real fossil binary, when one is on PATH. check-in: 0ca5133c62 user: [email protected] tags: trunk

* fix: bound delta.Apply growth per-command and cut ~5x allocation overhead (#81, #82)

Two related fixes to the same command loop in delta.Apply:

- #82: check len(output) against the declared target length after every copy/insert command, not just at the ';' terminator. A malformed delta with many copy commands could previously grow output well past both targetLen and maxDeltaTargetSize before anything noticed, since the only comparison happened at the end.

- #81: once a command has actually executed (proving the delta contains real work, not just an inflated header claim), upgrade output's capacity in a single step sized off len(delta) rather than relying on append's own ~1.25x large-slice growth factor to climb from the 64 KiB initial cap to the final size. That climb was landing near 5x the target size in cumulative allocation; the single upgrade restores near-exact sizing for honest deltas while keeping the tiny-payload/ huge-header attack closed, since a delta with no real commands never reaches the upgrade.

Added TestApply_LoopBoundsAgainstTargetDuringCommands (regression for #82: many copy commands exceeding a small declared target are rejected early, with allocation growth bounded near the declared target rather than the full command sequence) and TestApply_LargeTargetAllocationBounded plus BenchmarkApply_LargeTarget (regression/measurement for #81: a 1 MiB target now allocates ~1.06x its size instead of the prior ~5x).

Cross-checked against fossil 2.28 for wire-format compatibility; full non-dst suite and dst simulation suite (90 scenarios) pass.

* fix: address delta.Apply review nits from #81/#82

Rebuild the #81 regression test/benchmark around a many-small-copy delta shape (4096 copy commands into a 1 MiB target) instead of a Create()-derived single-big-insert delta. The single-insert shape allocates ~1.06x on both fixed and unfixed code (grows once), so it neither exercised the ~5x append-growth-climb regression nor would have caught a future reintroduction of it. Verified against a reverted delta.go: unfixed code now measures ~5.29x on this benchmark, fixed code ~1.06x.

Also move the len(output) > targetLen bound checks ahead of their append calls in both the copy and insert command branches, so an over-target command is rejected before its bytes are appended rather than after -- matches the #82 issue wording exactly, closing the (non-exploitable, since a single command's overshoot is already bounded and resident) gap the reviewer flagged as a nit. check-in: 355e7d9910 user: [email protected] tags: trunk

* manifest: create forumpost on demand, matching canonical rebuild

Crosslink's candidate query names the forumpost table unconditionally, but canonical fossil creates it lazily -- only once a forum artifact needs it -- and fossil rebuild drops it for a repository that never populated it. Crosslink now recreates the table itself before the sweep runs, matching canonical's on-demand behavior instead of erroring with "no such table: forumpost" against a rebuilt repo.

Audited the rest of the on-demand-in-canonical set (ticket, ticketchng) against the current schema: both are created eagerly by canonical at init and survive rebuild, so they don't share this problem.

Added TestCrosslinkForumPostTableDropped (drops the table directly) and TestCrosslinkAfterFossilRebuild (drives an actual fossil rebuild first) to cover the regression, and updated the existing fossilbin comparison test's comment now that the rebuild-last ordering is no longer required.

* manifest: visit crosslink candidates in delta-chain order

Crosslink swept candidates in ascending rid order, but content_deltify stores a blob's older versions as deltas against its newer ones, so delta chains run the opposite direction: high-rid root down to low-rid ancestors. An ascending sweep's first candidate of a chain therefore materializes the whole chain, and the rest are consumed arbitrarily far in the future -- the working set of a whole-repository sweep was the entire repository expanded (8.2 GiB for the Fossil SCM repository).

Candidates are now topologically sorted so a delta's base is always visited before its dependents (deltaChainOrder), matching Fossil's own rebuild_step shape: root first, each descendant one delta application from an already-expanded predecessor. The working set this sweep needs is now bounded by how many chains are in flight at once, not by repository size.

That reordering also changes the order artifact side effects get written in, since content expansion and artifact crosslinking share one loop. Most of what Crosslink writes turned out to already be a pure function of its own artifact (event/plink/mlink/tagxref-origin rows) and needed no change. Two things generalize across the whole plink graph rather than any one artifact and were genuinely order-sensitive:

- leaf table bookkeeping, which inserted the new checkin and deleted its parent -- correct only when parents are linked before their children, which delta-chain order no longer guarantees. - tag propagation, which ran once per checkin from its immediate parent -- reachable only to children plink already held at that moment.

Both are now full, order-independent repair passes (repairLeafTable, repairTagPropagation) run once after the sweep, mirroring Fossil's own manifest_crosslink_end + leaf_rebuild() shape: defer the order-sensitive work and repair it once rather than trying to preserve order through the sweep. tag.propagate's mtime-gated, plink-live-at-call-time design turned out to already be confluent -- replaying every self-declared tag origin once, in any order, converges on the same tagxref state -- so the repair is just doing that instead of depending on visiting order to get there implicitly. Tech-note event replacement and forum thread titles turned out to already have the same confluent shape and needed no change; left comments explaining why.

Also fixes a tag.PropagateAll crash on any origin with a NULL tagxref value (e.g. CrosslinkCluster's cluster tag), surfaced by the new propagation repair actually reaching those rows for the first time.

Regression coverage: a real three-generation checkin chain, forced into a genuine manifest delta chain, that would have left two of three checkins untagged and all three marked leaf under the old per-checkin propagation and incremental leaf bookkeeping.

* fix: address review nits on crosslink order-sensitivity and delta cycles

- deltaChainOrder now returns an error instead of panicking when the candidate delta graph fails to drain. The delta table is on-disk data that can arrive over sync from an untrusted peer, so a cycle (corrupt or hostile repo) should surface as an error rather than crash the process.

- snapshotDerived's digest now covers tagxref (rid/tagid/tagtype/ srcid/origid/value), and TestFossilBinaryReadsCrosslinkedRepo compares it against canonical fossil rebuild -- closing the gap the test previously called out by name ("tagxref is the notable gap").

- Added TestFossilBinaryReadsCrosslinkedRepoBranchAndMerge, exercising the order-sensitive paths incrementalHistory's single-branch history cannot reach: a feature branch diverging from trunk, and a merge commit whose own branch tag must come from its primary parent, not the merged-in one. event/plink/tagxref match canonical fossil row-for-row on this topology.

leaf and mlink are deliberately excluded from that comparison: the fixture surfaced that both have pre-existing gaps against canonical fossil (repairLeafTable counts any plink edge rather than only primary-parent edges; insertCheckinMlinks writes a row for every F-card rather than only changed ones) that predate this fix and are out of scope for #102/#103.

* fix: store received wire frames verbatim instead of re-encoding

Clone reproduced content correctly but not bytes: the receive path decompressed every cfile frame and re-compressed it with blob.Compress's unparameterized zlib writer, so a clone's blob.content never matched its source's — about 0.5% larger on the wapp corpus, purely from re-encoding frames that were already correctly encoded on the wire.

xfer's cfile decoder now hands back StoredBlob alongside the decompressed Content: the same bytes recast into Fossil's on-disk blob format ([4-byte BE size][zlib data]) without a second zlib pass — verbatim when the wire already sent that format (Fossil's send_compressed_file case), or the received zlib stream with a length prefix bolted on when it didn't (our own encoder's plain-zlib case). storeReceivedFile/storeDeltaContent/ storeResolvedContent and blob.StoreDeltaRaw thread that through via a new blob.EncodeForStorage helper, which writes the verbatim bytes when given and only falls back to Compress when there's nothing to preserve — covering both the clone/pull and server-push receive paths.

Locally authored content (never received over the wire) always takes the Compress fallback, so #71's deltification policy is untouched. The clone BUGGIFY corruption test drops storedBlob alongside the corrupted content it's exercising, so verbatim storage can't mask the corruption it exists to catch.

Adds a corpus-level regression test cloning a real multi-revision fossil repo and asserting sum(length(content)) matches between source and clone.

* test: use RequireFossilBin instead of removed HasFossil helper

testutil.HasFossil was removed upstream in favor of RequireFossilBin; this test's skip guard needed the same update to compile against current main. check-in: cc42ad8829 user: [email protected] tags: trunk

* test: route fossil-binary tests through one REQUIRE_FOSSIL_BIN helper

REQUIRE_FOSSIL_BIN=1 turns a missing fossil binary into a test failure instead of an invisible skip, but only internal/manifest honored it. The six other fossil-dependent sites each did their own exec.LookPath + unconditional t.Skip, so on those a run with no fossil was byte-identical to a passing one.

Add testutil.RequireFossilBin, which resolves the binary the same way FossilBinary does (FOSSIL_BIN override, then PATH) and either skips or fails based on REQUIRE_FOSSIL_BIN. Route testutil.NewTestRepo, both internal/delta CLI round-trips, the four internal/merge validation tests, the internal/xfer sanity check, and the ls-branch-tip repro through it, deleting their bespoke lookup+skip logic. CI already sets REQUIRE_FOSSIL_BIN=1 on the unit-test steps, so this makes all seven sites fail loud on absence while local runs without fossil stay opt-in.

* test: route every fossil-binary gate through RequireFossilBin

The first pass left six sites still deciding for themselves whether a missing fossil binary is a skip or a failure. Three of them (internal/sync's startFossilServer and its two integration tests, manifest's two rebuild tests, the root TestPhaseB_Integration, and hash's TestSHA1_FossilValidation) skipped unconditionally, so with REQUIRE_FOSSIL_BIN=1 and no fossil on PATH those packages still reported ok -- the exact silent pass the flag exists to prevent. The other three had hand-rolled copies of the flag check, which is the duplication that let the original drift happen.

All of them now call testutil.RequireFossilBin, and HasFossil is gone: it was a second, skip-only way to ask the same question, and it is what produced the sites that ignored the flag.

Also validate the FOSSIL_BIN override with LookPath, so a stale path reports "fossil not found" instead of surfacing later as a raw fork/exec error.

* test: resolve fossil binary via RequireFossilBin at remaining call sites

* fix: correct stale module path missed by rebase auto-merge

Three imports added by earlier commits in this branch predated the rebase target and had no corresponding line on main to conflict against, so git's auto-merge carried the old module path through verbatim. check-in: f804878bd3 user: [email protected] tags: trunk

* fix: degrade unparseable clone VERSION/SEQNO instead of erroring (#95)

parseClone routed both tokens through strconv.Atoi before the §8.1 digit-only rule ever got a chance to apply, so a token Atoi could not parse at all (not just one that fails digit-only) hard-errored the whole message instead of degrading gracefully. Carry the raw tokens forward as an UnknownCard on parse failure, matching how UnknownCard already handles a non-decimal clone_seqno, so a later card in the same message still decodes.

* fix: reject an oversized §4.1 declared length before trying raw zlib (#110)

xfer.Decode tried the unprefixed-zlib framing before ever reading the §4.1 container's declared length prefix. A peer declaring a length at or above 0x08000000 produces a prefix whose first two bytes present as a structurally valid zlib CMF/FLG header (RFC 1950), so a large declared length aliases as the start of a raw zlib stream and gets first refusal on a legitimate compressed container. Read the declared length whenever the body opens with a valid zlib header and reject it against MaxDecompressedBytes before attempting format 1 at all. check-in: 2b9694f525 user: [email protected] tags: trunk

The internal/hash import was added by the #93 T-card validation fix on a branch cut before the go-libfossil module rename merged. The rename's automated rewrite only touched import lines that existed at the time, so this newly-added import kept the old path and broke go build ./... on main after both merged.

CI's go build ./cmd/libfossil/ and go test $(go list ./... | grep -v /dst) didn't catch it: go list ./... drops a package that fails to compile from its output rather than failing the command, so the broken package silently dropped out of the test run instead of failing the build. check-in: 7764ecf6ff user: [email protected] tags: trunk

* fix: J card decodes value, stores name verbatim (#90)

The J-card parser had the escape direction inverted: it decoded the field name and exposed the value with its escape sequences intact. Both the spec (4.7.8) and canonical fossil (manifest.c) do the opposite -- defossilize only the value, compare the raw name.

Swap both the parse and marshal directions so the value round-trips decoded and the name stays verbatim.

* fix: T card escape-encodes the tag name on marshal (#91)

A tag/branch name containing a space was written verbatim, so 'my branch' produced 'T *sym-my branch *' -- a structurally valid but silently wrong card that re-parses as two tokens.

Align with canonical fossil: escape-encode the name on marshal and escape-decode it on parse, leaving the target token verbatim in both directions (4.7.16). TagCard.Name now holds the decoded form, so tagOrderKey reattaches the sign without re-decoding.

* fix: enforce two skipped T-card §4.7.16 checks on parse (#93)

parseTCard validated the sign and ordering key but accepted two malformed shapes from untrusted peers:

- a name that after the sign is entirely hex digits, indistinguishable from a hash target; - a target that is neither a valid artifact hash nor the literal '*'.

Reject both with errors (never a panic). hash.IsValidHash reuses the same target check the primary uuid path uses; the '*' and non-empty guards run first since it panics on an empty string.

* fix: T card decodes and encodes the tag value (#91), guard hex names on marshal (#93)

§4.7.16 applies the same escape rule to all three T-card tokens: the name and value are escape-decoded on parse and escape-encoded on marshal, while the target is stored verbatim. Only the name was handled; the value was passed through raw in both directions.

The read side was the real damage. `fossil branch new 'my branch'` writes

T *branch * my\sbranch T *sym-my\sbranch *

so importing an ordinary fossil repo with a spaced branch name produced a branch literally named "my\sbranch" through crosslink. Verified against the fossil binary: the artifact for that command now parses to value "my branch" and re-marshals byte-for-byte identical.

Marshal also gains the mirror of the hex-only tag name check parse already enforces. Without it a caller could build and store an artifact this repository could never read back; canonical fossil never emits one either, since it prefixes branch and user tags with "sym-". check-in: 0e579e24a4 user: [email protected] tags: trunk

* fix: correct go.mod module path to go-libfossil (#76)

The module directive still declared github.com/danmestas/libfossil after the repository was renamed to go-libfossil. pkg.go.dev/godoc scraping matches the declared module path against its hosting path, so the stale path broke SDK doc generation.

Rename the module path across all four in-repo modules (root, the two db/driver submodules, observer/otel, and docs/site) and every internal import, plus the release workflow, the doc-generation script, and the docs.

No tag has been published under the go-libfossil path yet, so the cross-module requires cannot resolve from the proxy. Add replace directives pointing the in-repo requires at their local directories so GOWORK=off builds and the CI matrix resolve without a published version. These replaces are ignored by downstream consumers and can be dropped once new-path tags are cut.

* Guard releases against replace directives; regenerate SDK docs

The go-libfossil rename bridges the in-repo modules with `replace` directives. Go refuses a main module that carries them, so a tag cut while they are present breaks `go install .../cmd/libfossil@<tag>` and every `@version` fetch of the driver submodules. Add a Release workflow step that fails the job before anything is published if any tracked go.mod still contains a replace directive, so the bridge is a CI-enforced TODO rather than a silent landmine. The step comment records what unblocks a release: re-tag the submodules with the corrected module paths, bump the root requires, then drop the replace blocks.

Regenerate the SDK reference docs. The previous pass only substituted URLs over the pre-existing generated files, leaving stale signatures (Repo.Timeline was still documented as taking LogOpts rather than TimelineOpts). Note that gomarkdoc must be run outside a git worktree — inside one it silently strips every source link.

Also reword the README rename note, which read as though the import path had not changed, and re-sort the ncruces require block. check-in: 6273a0750d user: [email protected] tags: trunk

deltifyBreaksLoop bounded its ancestor walk on deltifyMaxChainWalk (1,048,576), 64x the maxDeltaChainDepth (16,384) cap the read-path walks use, with no rationale for the difference. It is the same delta table over the same chains, so the smaller bound's reasoning applies verbatim.

Bound deltifyBreaksLoop on maxDeltaChainDepth and remove the now-unused deltifyMaxChainWalk constant. Behaviour is unchanged for any real corpus -- the bound only bites far beyond the deepest observed chain (2,546). check-in: 17724bfdfd user: [email protected] tags: trunk

* fix: never parse a compressed body as cards (#104)

A clone served by libfossil failed in libfossil's own client with

xfer: decode card 16303: xfer: empty line after split

on a response that contained 202 cards. The index, the card, and the fault were all fictional.

xfer.Decode tried three body framings in order -- raw zlib, the §4.1 4-byte-length container, then uncompressed card text -- and treated any decompression failure as "this is not that framing". decompressBounded failed a well-formed container purely because its decompressed length passed a 50 MB bound, so the fallback chain handed the still-compressed bytes to the card parser. The parser read compressed noise as cards for sixteen thousand iterations before one of them split into no fields, and reported that as the fault. Captured at offset 15,084,752 of the failing body, the "card" the parser was reading is DEFLATE output:

"...\xd9\x34\x50\xe7\xcf\xa0\xf8\xd5\xd5\x87l\xff\t\xde\xfb\xe3..."

Two changes. Once a body is recognized as a zlib stream, a failure to decode it is reported rather than reinterpreted -- oversize, truncated, or corrupt, the error now names the transport failure instead of an arbitrary card position. Both compressed framings are still attempted, since a length prefix can coincidentally parse as a zlib header, and uncompressed bodies still reach the card parser; only compressed ones no longer can.

The bound itself was also wrong, and is the reason this was reachable at all. §4.1 places no limit on the decompressed size of the container; the 50 MB was a local memory guard that sat below what this implementation's own server emits in one clone round, so two libfossil peers could not clone from each other even though each could clone from fossil. It is now 64 MiB and coupled to sync.DefaultCloneBatchBytes by a compile-time guard that fails the build, rather than the clone, if the budget is raised past half of it.

The defect predates the size-bounded clone batching of #100. #100 shrank rounds enough to hide it on the corpus that exposed it; it did not touch the decoder.

Regression coverage, in two layers. internal/xfer pins the decode behaviour directly: an oversize container, a truncated one, a large in-bounds one, and an uncompressed body. internal/sync adds the configuration no existing test covered -- libfossil serving and libfossil cloning across a real HTTP encode/decode, on a corpus of multi-megabyte incompressible artifacts that crosses the server's per-round budget. Every prior clone test was either served by fossil or wired through MockTransport, which passes *xfer.Message values across untouched and never encodes a wire body at all. Artifact size is the other half of the reproduction: this failed on wapp, whose artifacts are large, and not on pikchr, whose artifacts are small.

Verified against the wapp corpus: the round trip that failed now completes with 1038 of 1038 artifacts on both ends.

* fix: make the #104 fixture reach the bound, and pin the bound from both sides

Review found the flagship regression test passed against pre-fix code. It did. Its corpus was 21.6 MB spread over twelve artifacts, and pagination keeps every round near the 16 MB budget however large such a corpus grows, so the round never came near the 50 MiB bound it was supposed to cross. The 25-line rationale reasoned about crossing DefaultCloneBatchBytes, which is not the quantity that matters and never was; a permanent comment asserting a mechanism that does not exist is worse than no comment.

The quantity that matters is one artifact. A round is charged against the budget before each artifact and sends whole the artifact that crosses it, so a round carries budget plus one entire artifact of unbounded size. A single artifact larger than bound-minus-budget is the only corpus that can exceed the bound, and it is also the one path by which a round can still exceed the current bound -- so the fixture guards the live risk and reproduces the historical one with the same bytes. It is now one incompressible artifact of 7/8ths of MaxDecompressedBytes, sized against the constant so it tracks it, behind testing.Short.

Verified against a pre-fix tree with both test files dropped in:

TestCloneSelfRoundTripOversizeArtifact FAIL Clone: handleFile aad935c2...: UUID mismatch for received file: expected aad935c2..., got 501b4814...

which is the sharper end of this defect: pre-fix did not merely misreport, it accepted the compressed bytes as cards and produced an artifact whose content did not hash to its name.

The cheap end-to-end round trip is kept as a separate test and described honestly as a coverage plank -- the first test anywhere here to have libfossil decode a body libfossil produced -- rather than as a guard on anything.

Second finding, message.go: the claim that format 1 is the "Fossil HTTP sync wire format" is false. Every application/x-fossil body carries the §4.1 length prefix, request side and reply side, both through blob_compress. Format 1 matches nothing canonical emits and is kept as a tolerance, now labelled as one.

Its safety was also load-bearing on a constant with no stated relationship to it. For a container of decompressed length N, prefix byte 0 is N>>24, and a zlib CMF requires CM==8 in the low nibble, so no container under 128 MiB can present a valid header at offset 0. Raising MaxDecompressedBytes past that point would leave decompressContainer's prefer-what-decodes ordering as the only remaining defense -- a narrower margin than this code was written against, and #98 is exactly the change that might want a different bound. That is now a build failure rather than a trap, matching the guard already on the other side. Verified to fire at 0x08000000 and to permit 0x07FFFFFF.

Also: the MaxDecompressedBytes doc claimed to stop a hostile body from exhausting an embedding process's memory. It does not -- the transport reads the body whole before Decode sees it. It bounds expansion only, and now says so.

Test hygiene: the server handshake polls for a live listener instead of sleeping 200ms and dropping ServeHTTP's error, and the 32 MiB and 64 MiB xfer allocations skip under -short like their sync counterpart.

* fix: correct three claims about what the #104 guards cover

The fix is unchanged. Three statements about it were wrong, and a comment that misstates a mechanism is worse than no comment because the next reader trusts it.

The ordering claim was the serious one. decompressContainer was described as "preferring whichever framing actually decodes, so a spurious header match still loses to a real container". It does not. Format 1 returns immediately on success and format 2 is reached only when format 1 fails; the error comparison at the end selects which failure to report when both have already failed, and protects nothing when format 1 spuriously succeeds. Probed: a bare zlib body returns from format 1 with format 2 never attempted.

That makes the aliasing guard weaker than its comment claimed, and in a direction neither review had reached: the aliasing byte is the top byte of the length the PEER declared, and this decoder never reads that field -- it slices past it and counts bytes actually inflated. No value of MaxDecompressedBytes constrains what a peer declares. A container declaring 0x081D0000 while carrying 46 bytes presents a parsing zlib header at offset 0, measured.

What actually saves a real container in that case is the stream, not arithmetic: after two prefix bytes are eaten as a header, inflate must still parse the remaining two plus the real stream as DEFLATE and match its Adler-32. Measured `flate: corrupt input` at all three aliasing lengths probed, so format 1 fails, format 2 runs, and the container decodes correctly.

The guard is kept because it states something true and useful -- Encode writes uint32(len(raw)) as the prefix, so holding the bound below the threshold means no body libfossil emits can carry an aliasing prefix -- but it is now documented as an invariant on our output, not a defence of our input, with the airtight alternatives named and left to their own issues.

Second, "a single artifact larger than this bound cannot be cloned" is true but not the operative threshold. A round already holding bytes has that much less room, so the ceiling is the bound minus what the round accumulated first. Measured on this branch with a 58,720,256-byte artifact: 8, 12 and 15 MB of filler ahead of it all fail, and 17 MB passes because it exhausts the budget and pushes the artifact into a round of its own. Non-monotonic in the filler, which is worth knowing. That is #109, a defect this fix exposes rather than causes; the fixture's claim to guard it is withdrawn and it now says plainly that it covers the leading-artifact case only.

Third, the changelog's "pin the bound from both sides" overclaimed. The guards pin the bound against the batch budget and against the aliasing threshold. Neither certifies a round.

Rebased onto 8b6da9a. The internal/content -race failure reported last round was not a flake and not unrelated -- this branch predated #107, which rewrote available.go. TestIsAvailable_CyclicChainTerminates goes from exceeding its 30s budget to 0.06s across that rebase.

---------

Co-authored-by: Dan Mestas <[email protected]> check-in: c892c7a622 user: [email protected] tags: trunk

* perf: memoize delta-chain interiors during crosslink (#89)

manifest.Crosslink ran at roughly 1.4 events/sec on the Fossil SCM repository, which puts a 27,000-artifact clone at two and a half hours of CPU after the transfer has already finished. A CPU profile of a sweep over that corpus attributes 152.77s of Crosslink's 167.20s -- 91% -- to content.Expand, and 126.35s of that to the single blob.Load inside its delta-replay loop.

The cause is chain depth, not per-event overhead: not a transaction per event, not a missing index, not a full-table scan. content_deltify stores a file's older versions as deltas against its newer ones, so one file's history is a single chain -- 469 nodes deep on average across that repository, 1,032 for the blobs carrying artifacts, 2,546 at worst. Expand walked each chain from its root on every call and Crosslink calls it once per blob, so the sweep performed 31.7 million delta applications to expand 67,451 blobs.

Cache.Expand now stops the chain walk at the deepest ancestor it already holds and keeps every node it materializes on the way forward, and Crosslink runs one such cache for the sweep. Blob content is immutable once written and the candidate rids are snapshotted before the loop, so nothing a sweep does can invalidate an entry.

Every node handed to the cache is verified against the UUID its blob row is stored under before it is stored, because a cached interior is served later as if it had been expanded in its own right. The number of hash checks per sweep does not change: each blob is materialized once and verified once, where before each was expanded once and verified once. A plain Expand, with no cache, walks and verifies exactly as it did.

IsAvailable folds its two-statements-per-chain-node walk into one. It runs once per F-card of every check-in being crosslinked, over the same deep chains, and every SQL round trip there pays a WAL read-lock acquisition -- the round trips, not the work they do, are the cost.

No transaction boundary moves and no batching is introduced. A manifest that fails to crosslink still fails on its own, exactly where it did.

Measured on the Fossil SCM repository, 67,451 content blobs and 27,060 artifacts. Time for Crosslink to write its first 300 event rows: 221s before, 131s after. Over the expansion path in isolation, sweeping the same corpus in rid order: 22.9 blobs/sec with no cache, 50.1 at the 256 MiB budget set here, 414 at 1 GiB -- where the process needs 2.9 GB resident, which is why 1 GiB is not the number.

A bounded cache cannot close the gap on its own. Chains run from a high-rid root down to low-rid ancestors while the sweep visits rids ascending, so the first candidate of a chain materializes all of it and the remaining nodes are consumed arbitrarily far in the future; the working set of a whole-repository sweep in rid order is the whole repository expanded, 8 GiB for this corpus. Closing it means visiting candidates in delta-chain order, which is how Fossil's own rebuild avoids the cost, and which changes the order-sensitive state Crosslink writes -- leaf, tag propagation, tech-note event replacement, forum thread titles. That is a separate change.

TestFossilBinaryReadsCrosslinkedRepo covers the acceptance criterion this cannot self-certify: a repository whose relational tables were written through the accelerated path passes fossil 2.28's test-integrity, shows its tip in fossil timeline, survives fossil rebuild, and its event, plink, leaf and mlink rows match, column for column, what fossil derives from the same blobs.

* fix: bound and cycle-check the delta-chain walks (#89)

IsAvailable walked the delta chain with no cycle detection, to a bound of 100,000 against real chains whose deepest observed node count is 2,546. It is wire-reachable -- sync/clone.go, sync/handler.go, and once per F-card of every check-in Crosslink processes -- so a peer that plants a two-node delta cycle bought a 100,000-iteration query loop on every call, inside whatever process embeds this library.

Both halves are fixed. Each walk now carries a visited set and stops at the first repeat, which is the actual defence: a cycle costs its own length, not the bound. maxDeltaChainDepth replaces the old constant at 16,384, roughly six times the deepest chain measured on the Fossil SCM repository, and it is now a backstop for an acyclic-but-absurd chain rather than a number an attacker spends. walkDeltaChain, which the memoizing expansion path made hotter, previously had no bound at all and now shares the same one.

This is what made TestIsAvailable_CyclicChainTerminates sit at ~99% of its 30s budget on main and fail outright under -race here: the cyclic walk was running the full bound. It now finishes in about a second under -race, six runs out of six.

walkDeltaChain returns the content it stopped on instead of leaving expandChain to ask the store a second time. The truncation decision and the use of it were separate steps, so a concurrent eviction between them would send a delta row to blob.Load and hand raw delta bytes to the replay as full text. bindToName caught that and failed closed, but the caller saw a hash mismatch on an artifact that was not corrupt, and Crosslink's error path drops those silently. Unreachable while the cache is a single-goroutine local, but Cache is documented concurrency-safe and takes its mutex on every method, so the type promised something the expansion path did not deliver.

Cache entries are clipped to their length before being stored, so curSize is the bytes actually retained rather than an allocator-dependent approximation of them.

Test corrections: the corpus harness no longer closes the repository handle out from under the sweep goroutine when it returns at the event target; the sweep owns the handle and closes it. snapshotDerived now fails on an empty digest rather than comparing "" to "" and passing without examining a row. The fail-closed injector matches the merged chain-step statement specifically rather than any query mentioning delta, and says what it can and cannot prove now that the two lookups are one statement. The fossilbin comparison documents that it is not full-table equivalence, and which tables it leaves uncompared.

MEASUREMENTS, SUPERSEDING THOSE IN 2426591

The figures in the parent commit were taken before clipping, and its "221s -> 131s" pair predates the double-hash and IsAvailable fixes that landed in that same commit. Everything below was re-measured on the code as it ships, one run at a time on an idle machine, against the Fossil SCM repository (67,451 content blobs, 27,060 artifacts).

Crosslink to its first 300 event rows, cpuprofile on throughout:

original code 221.4s as shipped 103.9s 2.1x

Expanding the first 4,000 blobs, which isolates the expansion path:

budget throughput peak RSS entries held none 22.1 blobs/s 169 MB 0 0 256 MiB 52.0 blobs/s 1036 MB 543 255 MiB 1 GiB 468.3 blobs/s 2278 MB 15,842 1021 MiB

Clipping changed neither throughput nor memory. Crosslink to 300 events runs 103.9s clipped against 105.9s unclipped, peak RSS 1067 MB against 1050 MB, and the cache holds the identical 543 entries and 255 MiB at a 256 MiB budget either way -- at these sizes the buffers already arrive tightly sized. It is worth keeping as an accounting guarantee, not as a saving, and the comment on clip says so.

The earlier claim that capacity slack was why a 256 MiB cache reached nearly a gigabyte resident was wrong. Peak RSS tracks the no-cache baseline plus roughly twice the budget, because the cache is live data and the collector sizes the heap against it: 169 + 2x1021 = 2211 against 2278 observed at 1 GiB. That is why the budget stays at 256 MiB -- a gigabyte of cache costs over two gigabytes resident to clone a 72 MB repository. The shape of the curve did not move.

* docs: correct the claims around the chain bound and the RSS model (#89)

Three claims did not survive checking, and one bound was a node out.

maxDeltaChainDepth said it bounds "every delta-chain walk in this package". It does not: deltifyBreaksLoop is a third walk, under deltifyMaxChainWalk, which is 64 times larger. The rationale on the new constant applies to that walk just as well, but changing it would change deltify's shipped behaviour, so the sentence is narrowed to the two read-path walks it actually governs and points at the third rather than quietly covering it.

Two comments still named maxAvailabilityChainDepth, which no longer exists, and argued from IsAvailable having a bound "of its own" -- the framing of a world where the two walks had separate bounds. Against a shared constant that argument reads backwards. Both now name the shared bound, and the reason deltifyBreaksLoop keeps its own visited set and its own step cap is stated as the general rule it is: every walk over the peer-shaped delta graph carries both, and none of them relies on another walk's guard.

The retraction in the previous commit replaced one unsupported mechanism with another. "Peak RSS runs at roughly the baseline plus twice the budget" is a two-point fit whose worse point is the shipped one: it is 3% off at 1 GiB and 53% under at 256 MiB, where resident memory is four times the budget rather than twice. The measured table stays; the model is demoted to the direction the data supports, that peak RSS grows faster than the budget, with the fit's own weakness stated.

walkDeltaChain counted loop iterations where IsAvailable counts nodes visited, so the shared bound bit a node apart: a chain of exactly maxDeltaChainDepth+1 was unavailable to one walk and complete to the other. It now counts nodes, and TestChainBoundBitesAtSameLength pins the agreement at both boundaries -- nothing else ties the two loops together. check-in: 9864120a32 user: [email protected] tags: trunk

* fix: bound clone batches by output size, not artifact count (#88, #92)

The xfer server paginated a clone by a fixed 200-artifact count. A count cannot adapt to artifact size: 200 control artifacts and 200 large blobs produce wildly different messages, and neither corresponds to what the transport can carry. It capped a libfossil-served clone near 20,000 artifacts against our own MaxRounds=100 client, and left the response size unbounded in the other direction -- serving libfossil's own repository produced a single 44.9 MB round.

emitCloneBatch now accumulates the wire bytes it emits and stops once they reach DefaultCloneBatchBytes, mirroring canonical's `while( mxSend > blob_size(pOut) )`: the budget is tested before each artifact and the artifact that crosses it is still sent whole, which is what keeps the cursor advancing when one artifact exceeds the entire budget. The card header is charged alongside the payload so a repository of zero-length artifacts still paginates. Section 8.2 leaves the bound to the server ("clients MUST NOT infer count or size from range").

The budget is 16 MB rather than canonical's 5,000,000 max-download because this clone path emits fully expanded artifacts where canonical retransmits stored compressed form (Section 7.2) -- 80x to 220x the bytes on real repositories. Measured on libfossil's own repository (17,218 artifacts): 87 rounds and a 44.9 MB worst-case response before, 65 rounds and 24.7 MB after. Canonical's constant applied to expanded bytes would have needed 184 rounds, breaking clones that work today.

Adds the interop half: a canonical fossil 2.28 client cloning a generated corpus from `repo serve`, sized past the pagination boundary, checked for round-trip count, artifact parity with the source, and `fossil test-integrity`. The many-small-artifacts case is what discriminates the two bounds -- 2,056 artifacts clone in 3 round-trips under the size bound and 23 under the count bound. Corpus size is a parameter since the interesting behaviour is at the boundary. A missing fossil binary skips, or fails under REQUIRE_FOSSIL_BIN=1.

* review: state the ceiling honestly, fix the overhead comment

The size bound replaces an artifact-count ceiling with an expanded-bytes ceiling. It is a better ceiling and it beats the count bound on every corpus measured, but fossil- and sqlite-sized repositories still cannot be cloned from a libfossil server within MaxRounds. Record the full three-repository round table on DefaultCloneBatchBytes and say outright that only Section 7.2 removes the ceiling rather than moving it.

State the memory property as budget plus the largest artifact, not as a 24.7 MB ceiling. The crossing artifact is sent whole, so a repository holding one 2 GB artifact still emits a 2 GB round, and concurrent clones multiply it. 45 MB -> 25 MB is a large reduction of a pre-existing unbounded allocation, not a cap.

cloneCardOverheadBytes claimed to cover the hash and then len(uuid) was added on top, double-charging it. Drop the addend; 96 over-approximates the ~58-byte framing on its own and the slack absorbs the uncharged PrivateCard.

Also: smallBatch sets a one-byte budget rather than a batch size of one, so correct the comments that still describe the old knob; drop the post-loop multi-batch > single-batch check, which passes under both bounds and so discriminates nothing; and make serveRepo report a server that exited early instead of signalling into an uninstalled handler, where SIGINT's default action would kill the test binary outright.

* test: count clone batches on the server, not round-trips on the client

CI (fossil 2.23) and this machine (2.28) disagreed on an identical server: the multi-batch clone reported 1 round-trip against CI and 9 locally. The clone was correct in both cases.

fossil prints "Round-trips: N" carriage-return separated, once per round. 2.28 guards those prints behind bOutIsTty and adds a single final line for non-TTY output, so a captured clone yields exactly one occurrence holding the total. 2.23 has no such guard and emits every round into the capture, so the regex's first match was always 1 no matter how many rounds ran. Verified by cloning one repository from a real fossil server with both binaries: 2.28 prints "Round-trips:" once, 2.23 four times, first line 1 and last line 2.

So the assertion silently meant different things on different clients, and many-small-artifacts -- the only case that discriminates a size bound from a count bound -- was passing under 2.23 for the wrong reason, inert rather than green.

Count the clone batches the server emits instead. That is the property under test, it cannot be distorted by client-side reporting, and it is sharper: many-small-artifacts is 2 batches under the size bound against 22 under a reinstated count bound, where the round-trip numbers were 3 against 23.

This serves via sync.ServeHTTP directly so the handler can be wrapped. RepoServeCmd.Run is a five-line wrapper around the same call and TestRepoServeCmdCanonicalFossilClone still drives the command end to end. It also drops these subtests' use of the process-wide SIGINT. check-in: 37271d3f68 user: [email protected] tags: trunk

* Add libfossil version command

Sources the version string from runtime/debug.ReadBuildInfo (module version for go-install/tagged builds, falling back to a short VCS commit), with a -ldflags override for release builds where BuildInfo alone would collapse to "(devel)". Output is a fixed, single-line, four-field shape ("libfossil <version> <go version> <os>/<arch>") so a caller can rely on strings.Fields(Version()) unconditionally -- the motivating consumer stamps this into every emitted benchmark record.

No global --version flag: repo extract already owns --version as a command-scoped flag (source version to extract), and kong's global flags are visible in every subcommand's context, so a root-level --version would collide with it. version is a subcommand only.

Verified kong's existing parse-error path already exits non-zero (exitUsageError, 80) for unrecognized commands and flags against the pinned kong version -- added regression tests locking that in rather than changing behavior that was already correct.

* Reject malformed -ldflags version overrides at startup

buildVersion was used verbatim with no validation, so a whitespace- or newline-containing -X override silently broke Version()'s promised single-line, four-field contract: a space produced six fields instead of four, and a newline broke the single-line guarantee outright. The stated consumer is a benchmark harness parsing this line as a stable record, so a corrupted version field silently corrupts every record from that build in exactly the field meant to identify it.

A bad -X value is a build-time operator mistake, not a runtime condition, so it's rejected at package init rather than sanitized or worked around: sanitizing would silently mangle the operator's intent, and falling back to build info would report a version that is confidently wrong. Both would hide the mistake in the one field whose job is to be trustworthy. Failing the whole binary at startup surfaces a bad build the first time it runs, for any command, not just when someone happens to run `version`.

Also simplified versionFromBuildInfo: verified empirically (go1.26, `go build .` inside this checkout, with and without -buildvcs and inside/outside a VCS checkout) that Main.Version is only ever the uninformative "(devel)" in exactly the cases where vcs.revision is also absent from BuildInfo.Settings -- whenever a VCS revision is available, Go has already folded it into Main.Version as a pseudo-version. The separate vcs.revision fallback loop was dead code that could only ever duplicate a hash already present in Main.Version; removed it along with the now-unused shortCommit helper. check-in: 725c694cb9 user: [email protected] tags: trunk

* fix: remove StatusOpts, MergeOpts, CheckoutOpts — no call sites

No function in the package accepts any of these three types, and nothing constructs one anywhere: Checkout.Status() takes zero arguments, Repo.Merge takes positional string arguments, and CheckoutOpts had no construction site at all. A public type with no call site documents a capability that does not exist -- a consumer reading MergeOpts{Strategy: ...} in the docs could reasonably conclude a strategy-selecting merge API exists; it never did.

Repo.Merge and Checkout.Status keep their current signatures. An options-struct refactor for Merge was considered and declined.

CheckoutOpts.Force was also a third same-named Force field in this package, alongside the already-removed UpdateOpts.Force (#58) and the real, fully-wired ExtractOpts.Force -- removing it also removes that readability hazard. ExtractOpts.Force and its plumbing (checkout.go Extract -> internal/checkout.ExtractOpts -> extract.go's Force check) are untouched.

* docs: match em dash style in CHANGELOG breaking-change bullet

The new StatusOpts/MergeOpts/CheckoutOpts entry used ASCII double hyphens where the adjacent CloneOpts and UpdateOpts.Force bullets in the same file use a real em dash. check-in: 1a33f22119 user: [email protected] tags: trunk

* fix: clone_seqno is server-to-client only (#74)

The clone client transmitted a `clone_seqno` card to carry its pagination cursor. That card travels the other way: canonical's page_xfer() only ever writes it (src/xfer.c:1571) and has no parser for it, so a real fossil server answered `bad command: clone_seqno N` and the clone died on round two. Repositories that fit in a single batch never reached the pagination path, which is why wapp and pikchr cloned while fossil and sqlite did not.

The request cursor belongs on the clone card itself — `clone VERSION SEQNO`, read from token 2 at src/xfer.c:1553 — and the response cursor belongs on `clone_seqno`, which the client only consumes. Give each direction a single owner:

- The client sends its cursor on the clone card and never emits a clone_seqno card. It already consumed the server's card correctly; it now also rejects a negative value, matching src/xfer.c:2769. - The server reads its cursor from the clone card and the CloneSeqNoCard parser is gone, so there is exactly one way to set the cursor rather than two.

Aligning with canonical also corrects the cursor's meaning. Ours was the last rid sent, queried with `rid > cursor`; canonical's is the rid of the next blob to send, looped inclusively with `while( seqno<=max )`. Now that the value crosses to a real server it has to mean what canonical means, so emitCloneBatch queries `rid >= cursor` and reports the first rid it did not send, or 0 once the snapshot is exhausted. That collapses the old more/lastSentRID pair into a single nextRID and keeps the truncate hook honest by re-owing the dropped rid instead of skipping it.

TestCloneMultiBatchAgainstCanonicalServer drives the client against a stand-in server that implements the canonical contract and rejects a client-sent clone_seqno the way fossil does; it reproduces `bad command: clone_seqno` against the old client. TestHandleCloneCursorFromCloneCard pins the server half, including the inclusive cursor.

Verified against fossil 2.28 over a loopback server: fossil and sqlite now transfer every artifact (67,499 and 130,994 blobs), and wapp and pikchr still clone in 3 rounds.

* fix: conform clone cursor validation to the spec in both directions (#74)

Review caught two rules applied to the wrong side of the wire — the same axis error the original bug was.

The client rejected a negative clone_seqno as fatal, citing src/xfer.c:2769. That line is the server's rule for the SEQNO on a clone REQUEST. The reply parser is governed by §3.2 and §8.2: NEXT is recorded only when it is decimal, and a visible but non-decimal NEXT is a defined receiver-tolerance exception that leaves the recorded sequence unchanged and does not stop parsing later reply cards. Aborting the clone was the one thing the spec rules out. The rule belongs in the decoder, where a non-decimal NEXT now decodes to an unknown card: the token is preserved, no consumer records it, and following cards still parse. strconv.Atoi accepted "-1" off the wire, so this was reachable, not theoretical.

Conversely the server clamped `clone 3 0` to a cursor of 1 where §8.1 mandates a fatal: clear accumulated output, roll back, emit `error invalid clone sequence number`, and parse no later request card. Nothing is written before processDataCards, so returning early is what the rollback amounts to. Seeing this case at all needs a presence flag, because `clone` and `clone 3 0` both parsed to SeqNo 0 and only the former is exempt. §8.1 also keys on a digit-only SEQNO and withholds the fatal from wrong arity, a VERSION below 2, and a SEQNO that fails digit-only recognition, so the flag records digit-only presence rather than merely arity.

Both card rules now turn on the spec's `decimal = 1*DIGIT` production via a shared predicate instead of on whatever strconv.Atoi tolerates.

Also promotes emitCloneBatch's `count > 1` truncate guard to a documented liveness constraint. Under the old exclusive `rid > cursor` semantics `count > 0` was safe; under the inclusive cursor, dropping the only card in a batch reports a cursor equal to the one we entered with and the client re-requests forever. A check before the reply card now names a stalled round at its source instead of letting it surface as an opaque MaxRounds failure, and the cursor precondition became an error return rather than a panic — it is enforced ~350 lines away in a network-facing handler.

Records the deliberate asymmetry that our server ignores an inbound clone_seqno where canonical answers `bad command`, and renames TestHandleCloneSeqNo, which no longer sends that card.

* fix: name the clone SEQNO flag for the shape it records (#74)

HasSeqNo described presence, but the field records whether the SEQNO token is digit-only: `clone 3 -1` carries a SEQNO and sets it false. §8.1 keys its fatal on that shape rather than on the parsed value, which is why a plain int cannot stand in for it. SeqNoIsDecimal says so.

Reading it as presence would also collapse `clone 0 0` — a distinct wire form that parses to the same Version and SeqNo as a bare `clone` — back into `clone` on re-encode. encodeClone already guarded against that; a round-trip test over every clone wire form now pins it, and fails on exactly that input when the guard is removed.

Also records why the §8.1 rollback is currently vacuous: no card handled before processDataCards writes anything, so returning early leaves nothing to undo. Nothing enforces that ordering, and §3.5's ci-lock would break it, so the dependency is written down next to the code that relies on it.

Corrects a test comment that claimed a continuation property it did not exercise — the malformed card sits in the trailer, so nothing follows it. That half of §8.2 is covered by the follow-on card in TestDecode_CloneSeqNoNonDecimal. check-in: f2fa2cc751 user: [email protected] tags: trunk

* Enforce card occurrence classes and intra-run ordering

The deck parser enforced card-type ordering and nothing else. Each of the 14 single-occurrence card types was handled by a case that assigned, so a second occurrence silently overwrote the first, and the repeatable types were appended with no sort check at all. draft-fossil-artifact-format-00 §4.5.1 and §4.5.2 define both rules; implement them.

Duplicate rejection covers A, B, C, D, E, G, H, I, K, L, N, R, U and W. The guard is separate from the card-type check because that check permits equal consecutive letters. F, J and M gain a strictly-ascending check against the preceding entry, T the two-level (name, target hash) rule with the full sign-prefixed name as primary key. P and Q deliberately get neither: P order is semantic — primary parent first — and Q is unordered-repeatable. P also now appends across cards instead of replacing, which is what §4.5.1 means by repeatable. Z belongs to neither class, so it is accepted with no duplicate or last-card rule; the authoritative checksum is the trailing 35 bytes VerifyZ already verified.

All four ordering checks route through one comparator, deck.Compare, which is §4.5.3: unsigned octets, locale-independent, no normalization, no case folding. §4.5.3 exists because the same ordering underlies the delta-manifest merge walk and baseline binary search of §8.2, so a second copy would resolve a delta manifest wrongly rather than merely fail validation. The F check runs on the decoded filename, per §4.5.2.

Malformed input is reported through the error return, never a panic -- these bytes arrive from peers.

Two writer-side defects surfaced, both of which let this package emit artifacts its own parser now rejects. Marshal emitted J cards in caller-supplied order while already sorting F, M and T; it now sorts them. Marshal also sorted T by the concatenation name+target, which lets the target's leading hex digit compete with the tail of a longer name and reorders tags whose names are prefixes of one another; it now uses the two-level key. The F and M sorts and the R-card file enumeration route through deck.Compare as well, the latter per §6.3.

Closes #70

* Order T cards by decoded name and type the Z payload

Review found two ways the first pass diverged from canonical fossil in the accept/reject direction.

The T ordering key used the raw wire name. §4.7.16 says the name is escape-decoded and that the primary ordering key is the full name token including its leading sign character; manifest.c defossilizes the name before the ordering compare. Because "\s" begins 0x5C and a space is 0x20, the two forms order oppositely, so a run like

T *sym-a\sb aaaa... T *sym-a-c bbbb...

is valid, fossil accepts it, and we rejected it. Branch names may contain spaces, so this is an ordinary tag, not a corner case -- the same post-decode trap already handled for F. tagOrderKey now builds the decoded key and both the parser check and the marshaller's sort call it, so the two cannot drift apart.

The Z card accepted any payload. Accepting a Z line inside the body is right -- §4.5.1 puts Z in neither occurrence class and treats an earlier Z as content covered by the final checksum -- but §4.7.19 still fixes the payload at exactly 32 hexadecimal characters. Validating nothing left an arbitrary-byte channel in an otherwise fully typed grammar and would have stored and forwarded artifacts canonical refuses. Upper-case digits are accepted, matching fossil's validate16.

Also: the T sign prefix is now checked against {+,-,*} per §4.7.16, both because canonical checks it and because an unvalidated sign byte >= 0x80 would UTF-8-expand to two bytes when the ordering key is assembled and silently corrupt the comparison. Two stale §4.7.20 citations corrected to §4.7.16, and the two-level T test gained the prefix-name case that actually distinguishes a two-level comparison from a concatenated one.

* Cite the normative grammar for the case-insensitive Z token

The Z payload's case-insensitivity was justified by inferring it from the word "hexadecimal" in §4.7.19. §6.1 states it normatively: md5-token = 32hexdig-ci, a parse-accept grammar admitting both 0-9a-f and A-F. Cite that instead, and record the split it implies -- the structural check is case-insensitive while §6.2 keeps verification byte-exact against the lowercase form.

Also soften deck.Compare's claim that the delta-manifest merge walk relies on F ordering to the future tense parse.go already uses: the merge walk is a map overlay today with no comparator to share. check-in: 5036cae392 user: [email protected] tags: trunk

* Deltify predecessors on the commit path

Every locally-created artifact was stored whole. The delta table was in the schema and stayed empty, so a repository grew with the number of commits rather than the amount of change. blob.StoreDelta existed and worked but had no caller on the commit path; the only delta-store in use was blob.StoreDeltaRaw, from the sync receive path, for deltas that arrive pre-encoded. The write path never made a deltify decision at all.

Port canonical Fossil's content_deltify (2.28 src/content.c:852-947) as content.Deltify. Fossil deltifies backwards: a new artifact is always stored whole and its PREDECESSOR is rewritten as a delta against it, so the tip of every chain is full content and reading the newest version of a file stays a single blob load. Chain depth becomes an artifact's age in revisions rather than something that compounds, because an artifact already stored as a delta is never re-deltified.

blob.StoreDelta turned out to be the wrong primitive to wire up: it inserts a new row as a delta, whereas the policy needs to rewrite an existing row. It is left alone. content.Deltify lives in the content package because it needs Expand, which blob cannot import.

The policy is stated once, in content.Deltify, with each rule's canonical line cited: the 50-byte floors, the 75% ratio, the never-redeltify rule that bounds depth, the private-into-public refusal, and the loop break that undeltas the source and then still declines the pairing. Two call sites pass pairs, mirroring canonical's: insertMlinkRow for file content (manifest.c:1562) and Checkin for the parent manifest (checkin.c:3234). insertMlinkRow is shared by the fresh-commit and crosslink paths, so file deltification cannot drift between them.

On a 12-revision, 4-file history: 60 artifacts, 44 delta-encoded (73%), p50 chain depth 4, max 11, and 23,994 content bytes measured as sum(length(content)) -- down from 58,973 with everything whole. A repository produced this way passes fossil 2.28 test-integrity (60/60 blobs, 0 errors) and fossil rebuild, which re-derives every table by walking the chains.

* Bound the deltify ancestor walk, require a transaction, harden tests

Review follow-ups on the deltification policy.

Bound the ancestor walk. It was a bare loop with no cycle guard: a sync peer can shape a cyclic delta graph, because blob.StoreDeltaRaw records delta(rid,srcid) links from the wire without a cycle check, and an unbounded walk would spin inside the write transaction. Guard with the seen-set idiom content.walkDeltaChain already uses, plus a step cap. In the current ordering IsAvailable runs first on both rids and follows the same links under its own depth bound, so Deltify declines before the walk begins -- but that protection is incidental to a different question, so the bound belongs to the loop that needs it. Both behaviours are now pinned: the walk reports a cycle, and Deltify declines an ungrounded source.

Move both size floors ahead of the walk. Canonical returns on a sub-50-byte target (content.c:877-885) before entering the candidate loop, so it never reaches the content_undelta on :902. Running the walk first meant a small target whose successor chained back to it would inflate the source to full content and then decline anyway -- a storage regression and exactly the divergence from canonical's layout the surrounding comment claims to avoid.

Take *db.Tx instead of db.Querier. The rewrite is two statements, and a blob holding delta bytes with no delta row expands as though it were full content: silent corruption rather than a detectable error. Canonical wraps the pair in db_begin_transaction (content.c:938-941); the signature now makes that non-negotiable at compile time. Both call sites already passed a transaction.

Three tests were named for behaviours they did not pin, and each is now verified to fail when the behaviour is reverted:

- The 75% ratio test had no assertion at all; deleting the rule left it passing. It now asserts the pair is declined and stays whole. - The loop-break test could not distinguish declining from proceeding, so the undelta-then-proceed behaviour corrected in the previous commit was untested. It now asserts the target is left whole. - The manifest test checked only "at least one", so a regression from 11-of-12 to 3-of-12 passed. It now checks each index and names the manifest that stopped being encoded.

Also cover the source-side 50-byte floor, which the tiny-artifact test could not reach because it made both artifacts tiny and returned at the target check first, and assert that Undelta restores blob.size while Deltify leaves it alone (content.c:761 vs :935).

Gate the fossil-binary test on REQUIRE_FOSSIL_BIN, set in CI. A skip is invisible without -v, so a run with no fossil binary was byte-identical to a pass -- for the one criterion our own tests cannot substitute for.

Document that blob.StoreDelta is not the commit path's primitive. It inserts a new row as a delta; deltifying rewrites an existing one. It has no non-test callers and the package doc advertised it as the delta-storing entry point.

* Document what the deltify loop-safety argument rests on

Two comments, no behaviour change.

The IsAvailable(srcRid) guard is ours, not canonical's: content.c guards only content_is_available(rid) and then calls content_get(srcid) unguarded. It is also load-bearing beyond phantoms, because it screens out the cyclic delta graphs that would otherwise reach the ancestor walk. Say both, at the guard.

And say at the walk why its seen-set never fires in practice, so that someone reordering the prologue meets the reason rather than concluding the bound is dead code. The note lived only in the test file, which is not where that reader is looking. check-in: 2368cad91a user: [email protected] tags: trunk

* cli: add repo serve command wiring the xfer HTTP server

Repo.ServeHTTP has always been public, but nothing in the CLI called it, so a repo created or cloned with this tool could only ever be a clone leaf. Add `libfossil repo serve` as a thin adapter: open the repo, build the bind address, translate process interrupt into the context cancellation ServeHTTP already honors, and call it. No new server or shutdown logic.

Binds to 127.0.0.1:8080 by default; --addr overrides host and/or port. Verified against a stock fossil client (canonical binary, not just this library's own transport) cloning over an actual TCP round trip.

* cli: address review nits on serve interop test

Document why TestRepoServeCmdRun and TestRepoServeCmdCanonicalFossilClone are allowed to send a real SIGINT to the test process (no t.Parallel(), no competing signal handlers in package cli_test), so a future test in this package doesn't silently break that assumption.

Also log the canonical fossil binary path and version the interop test actually exercised. A skip and a pass read identically in a non-verbose CI log; if the apt install step that puts fossil on PATH is ever removed or reordered, this line is what would tell us the test stopped proving anything instead of just staying green. check-in: 7572f7d420 user: [email protected] tags: trunk

* fix: distinguish absent, empty, and valued U-cards in manifest parsing

Manifest parsing collapsed a wholly absent U-card and a present-but-empty U-card into the same stored value (empty string), so every crosslink call site that wrote the check-in user into event/attachment/forumpost rows lost the distinction between "no login recorded" (should be SQL NULL) and "login explicitly blanked" (canonical fossil resolves this to the literal "anonymous" at parse time — see src/manifest.c:1008-1016).

Change deck.Deck.U from string to *string so the three states are representable: nil (absent), non-nil "anonymous" (present-empty, resolved in parseUCard the same way fossil's own parser does), and non-nil with a value (present). Resolving once in the parser means every downstream crosslink/rebuild call site can keep binding d.U directly — Go's database/sql converts a nil *string argument to SQL NULL and dereferences a non-nil one automatically, so no per-call-site special casing is needed. The type change itself makes the old ambiguous string-binding path a compile error everywhere Deck.U is constructed, closing off the drift that caused issue #29.

Added deck.Str, a small pointer-literal helper, since Go can't take the address of a string literal directly; existing tests that built Deck{U: "..."} now go through it.

* fix: review nits — whitespace U-cards, rebuild coverage, tag.AddTag, naming

Four follow-ups from review of the U-card presence fix:

- deck.parseUCard now trims args before the emptiness check, matching canonical fossil's next_token() (skips leading whitespace) and every sibling card parser in this file. Previously "U \n" stored literal whitespace instead of resolving to "anonymous". Added a whitespace-only case to TestParseUCardStates.

- Added TestRebuild_UCardPresenceStates covering the rebuild path (rebuild_manifest.go's rebuildCheckin), mirroring TestCrosslink_UCardPresenceStates. The binding site was already correct (same nil-to-NULL pointer semantics as crosslink.go) but had no dedicated regression test.

- tag.AddTag now panics on an empty opts.User, matching the panic idiom CheckinOpts.User already establishes. Previously a tag created with an empty User silently crosslinked to user=''; that asymmetry with CheckinOpts was a wart worth closing rather than documenting as intentional.

- Renamed deck.Str to deck.User: the name should say what the helper constructs (a Deck.U value), not how it's implemented (a pointer take). check-in: 527759acd5 user: [email protected] tags: trunk

* fix: store delta content against a phantomized base instead of discarding it

A delta artifact arriving before its base is normal steady-state behavior during a Fossil transfer, not a fault. The receive path used to discard such deltas outright and phantomize the target instead, so a phantom only ever cleared when its base happened to already be present -- resolution advanced one delta-chain hop per round, making clones of deep-history repos provably unable to converge within MaxRounds.

Match canonical Fossil's content_put_ex/rid_from_uuid model: resolve the delta's source via a create-phantom-if-missing lookup, persist the delta bytes unconditionally (reading the target size straight out of the delta header, so the base never needs to be readable to store it), and record the delta-to-source link immediately. Availability resolves lazily via the existing content.IsAvailable chain walk and manifest.Crosslink's deferred retry -- both already correct and untouched here.

storeReceivedFile is the single call site shared by clone, sync-pull, and server-push, so fixing it here also closes out the delta-before-base sentinel handling on the pull and push paths (issue #57, folded into this fix): neither path needs to special-case the condition anymore because it stops being an error.

Verified against the pikchr corpus via a loopback fossil server: rounds dropped from 100 (exhausted) to 3, files received from 63,155 (22.2x) to 2,980 (1.05x), and the phantom table empties out completely.

* fix: converge delta storage on canonical, close integrity gap, harden wire parsing

Three defects from review, all in the raw-delta path added for the delta-before-base fix, all reachable from an untrusted peer:

1. delta.getInt accumulated v = v*64 + digit with no overflow check. A twelve-byte header of maximum-value digits wraps to exactly math.MaxUint64, which casts to int64(-1) -- the same sentinel blob.size uses for "phantom". That silently corrupted the row: real content reporting as a phantom, with no phantom-table entry to make it re-requestable. getInt now rejects overflow instead of wrapping, and parseTargetLen (shared by Apply and OutputSize) bounds the parsed target length to a sane maximum before it ever reaches an allocation or a size column.

2. StoreDeltaRaw panicked on a zero-length delta payload. storeReceivedFile guarded payload == nil but not len(payload) == 0, and nothing between wire decode and storage checked for it -- a peer sending an empty delta file card crashed the process. Wire data gets an error return, not an assertion; OutputSize and StoreDeltaRaw are now consistent with delta.Apply's existing "empty delta is an error" contract.

3. The receive-time split (store raw only when the base happens to be unavailable, otherwise eagerly expand and verify) left a gap an attacker chooses the timing of: withhold the base, and a delta claiming any UUID goes down the unverified raw path. Nothing downstream checked it either -- content.Expand applied deltas with no hash check, and content.Verify was reachable only from repo rebuild and one integration test.

Closed by converging fully on canonical's model: every delta is now stored raw and linked via a create-phantom-if-missing lookup, regardless of whether its base is currently available -- receive time no longer branches on availability at all. Verification moves to the one choke point every expansion path already goes through: content.Expand now hashes its result against the row's claimed UUID before returning it, catching a mismatch regardless of which caller or how the chain assembled. content.Verify becomes a thin wrapper over the same check.

Storage-layer and end-to-end regression tests added for all three, including the exact claim-one-UUID/ship-a-delta-for-another/withhold-the- base reproduction, guarded against degenerating into a no-op (delta must actually compress, base must be genuinely absent, rejection must be a real error rather than merely different bytes).

Re-verified against pikchr (2,842 artifacts) and wapp (1,039): pikchr converges in 3 rounds, 2,980 files, 0 phantoms; wapp converges in 3 rounds, 1,115 files, 1 phantom outstanding (pre-existing, unrelated -- clone.go has no DB-phantom rescan the way the pull path's loadDBPhantoms does; separately tracked). On-disk repository size is close to but slightly smaller than storing everything as raw delta bytes should produce (pikchr 6.81MB -> 6.73MB, wapp 5.64MB -> 5.46MB): zlib on full text already captures most of the redundancy an explicit delta would, so the win here is mainly integrity and convergence, not size.

* fix: bound delta allocation to actual work, validate delta source hex

Two defects from round-two review, plus a stray comment.

1. delta.Apply preallocated its output buffer directly from the delta header's claimed target length (make([]byte, 0, targetLen)). That claim only had to clear maxDeltaTargetSize (4 GiB) to be trusted as an allocation size, so a handful of wire bytes declaring a near-4 GiB target triggered an allocation sized to the claim, not to any work actually done. Harmless as a one-off parse spike; durable once a delta can be stored unverified (see blob.StoreDeltaRaw, this branch) -- a hostile peer plants one such delta and every later Expand of that row repeats the attempt. The initial capacity is now a small fixed bound; append grows it normally for legitimately large content instead of trusting the header's own claim about itself.

2. storeReceivedFile validated the target UUID's hex-ness (hash.IsValidHash) but not the delta source's. The blob table's CHECK constraint only bounds uuid length, not hex-ness, so a non-hex string of valid length passed straight through to blob.StorePhantom and became a permanent, unfillable row -- one the pull path's loadDBPhantoms would then re-request every sync round, forever. deltaSrc now gets the same check as the target uuid, at the same place.

Also deleted a stray doc-comment line left over from an earlier refactor.

Regression tests for both, RED-verified against reverted code before being fixed: the allocation test had to be corrected mid-way through -- delta headers encode their integer in Fossil's own base-64 alphabet, not decimal, so an initial fmt.Sprintf("%d", ...) fixture parsed as a different (still large but differently-bounded) value and passed vacuously; appendInt is the real encoder and is used directly now, with a round-trip sanity check against OutputSize guarding the fixture itself. check-in: 89535a6c42 user: [email protected] tags: trunk

* checkout: re-stat executable bit at commit time, not just on changed files

Change scanning compares content hashes, so a file whose only edit is a chmod (owner-execute bit gained or lost, no content change) is never flagged as changed. vfile.isexe then carries forward whatever was captured at the last Add/Commit, and the resulting F-card silently keeps the stale permission.

Mirror canonical Fossil's belt-and-braces approach (src/checkin.c): manifest generation re-stats each committed file's live on-disk mode independently of whatever change scanning recorded, using the existing modeIsExecutable predicate (added for the prior executable-bit fix) rather than trusting a cached flag a second time. This runs for every file with a vfile row that is part of the commit, not only ones flagged as content-changed, so a chmod-only edit is now captured.

Applies only to files actually present in vfile, leaves symlink markers alone, and is a no-op on Windows where os.FileMode carries no owner-execute bit.

* checkout: skip missing files during commit-time perm re-stat

restatCommitPerms Stats every tracked file that is part of the commit, and a Stat failure on any of them aborted the whole commit. That regressed a case the mode-change fix never meant to touch: a tracked file deleted from disk without Unmanage, with an unrelated file modified, used to commit successfully and carry the missing file's last-committed content forward — after the fix it failed the commit outright, on the implicit "commit everything" path in particular.

Skip only os.IsNotExist and carry the existing Perm forward unchanged, matching pre-fix behavior exactly. This fix is about mode changes being missed, not about detecting stale tracking — that is a legitimate feature canonical Fossil also checks for at commit time, but it deserves its own issue rather than arriving as a side effect here. Any other Stat error (permission denied, I/O fault, ...) still fails the commit. check-in: 094abd4b4e user: [email protected] tags: trunk

* fix: give Timeline its own result type so an Ancestry cursor cannot compile into it

LogEntry (Ancestry's result type) carried a Cursor field that was always the zero value on Ancestry entries — structurally identical to a legitimate Timeline first-page call. Feeding it into TimelineOpts.After silently paginated from page one forever with no error; documenting the field as invalid on Ancestry entries did nothing to stop it, because the zero cursor cannot be told apart from a valid first-call state even in principle.

Split the shared type instead of adding a runtime guard: LogEntry loses Cursor entirely and remains Ancestry's result type, while a new TimelineEntry (embeds LogEntry, adds Cursor) becomes Timeline's result type. An Ancestry entry's cursor can no longer be obtained at all, so passing one to Timeline is now a compile error rather than a documented hazard. Applied the same split to the internal manifest.LogEntry / manifest.TimelineEntry pair Timeline and Ancestry are built on.

Breaking change; CHANGELOG documents the migration.

* doc: warn against hand-assembling TimelineEntry from an Ancestry LogEntry

Review found the one remaining escape hatch: TimelineEntry{LogEntry: someAncestryEntry} compiles with a zero-value Cursor, which Timeline reads as "start from newest" rather than an error. No library path produces that value — it requires deliberately embedding an Ancestry LogEntry into a hand-built TimelineEntry — but flag it at the type so whoever does that gets the warning where they're typing. check-in: ad09ba3d97 user: [email protected] tags: trunk

Both were accepted by the public CloneOpts but never forwarded to the internal clone path, so setting either silently did nothing. The internal clone options struct has no matching fields at all, so wiring them would mean designing semantics first — is a caller- supplied code a validation assertion or an identity override? — and no caller has asked for either. Delete them rather than guess.

CloneResult.ProjectCode and CloneResult.ServerCode are unrelated and unaffected: they are populated from the internal clone's return value (the remote, via protocol negotiation), not from caller options, and continue to work exactly as before. Consistent with the same call made on UpdateOpts.Force. check-in: 9776c25462 user: [email protected] tags: trunk

Force was accepted by the public UpdateOpts but never forwarded to the internal update path, so setting it silently did nothing. Delete it rather than guess at forcing semantics; a real implementation can be designed and added later as a deliberate, wired field if a caller needs it. Rides along with the UpdateResult signature change already landing on Update in this cycle. check-in: cd674f058b user: [email protected] tags: trunk

* fix: populate mlink parent columns on the Crosslink check-in path

insertCheckinMlinks (the xfer/Crosslink ingestion path) wrote mlink rows with only (mid, fid, fnid) set, leaving pmid/pid NULL. Fossil's compute_fileage filters mlink.fid != mlink.pid, and in SQLite <int> != NULL evaluates to NULL rather than true, so every row written by this path was silently dropped from ancestry-walk queries - `fossil ls -R <repo> -r <rev>` returned empty for any commit that arrived over the wire, even though `fossil open` worked fine (it reads the manifest blob directly and never consults mlink).

Root cause: the two check-in paths that write mlink both needed to implement the same three-case pid convention (src/manifest.c:1668-1679 in canonical Fossil - carried-over/added/added-by-merge) and had drifted. insertMlinks (the direct check-in path) only handled two of the three cases and ignored merge parents entirely; insertCheckinMlinks didn't populate the parent columns at all.

Factor the convention into one place (internal/manifest/mlink.go): resolveMlinkParent implements the pid/pmid rule, insertMlinkRow wraps it and also fills pfnid (rename tracking) and mperm (permission encoding), which neither path wrote before. Both insertMlinks and insertCheckinMlinks now call insertMlinkRow, so the two write paths produce indistinguishable rows and cannot drift apart again.

insertCheckinMlinks also stopped skipping F-cards with an empty UUID (the delta-manifest deletion marker) - it now writes a row with fid=0, matching canonical's "deleted files have mlink.fid=0" and letting deletions participate in ancestry queries the same way Fossil's own writer does.

Un-gates ls_branch_tip_repro_test.go (previously untracked, guarded behind LIBFOSSIL_LS_BRANCH_TIP_REPRO) as a permanent regression test, and adds explicit coverage in internal/manifest/mlink_test.go for all three pid cases (normal add, merge add, delete) on both write paths.

* test: pin the deleted-file pid to the parent's exact file rid

Strengthens the deleted_file_gets_fid_zero subtest added for #29: it was only checking pid != 0, which would pass even if resolveMlinkParent picked the wrong nonzero value. Compare against the primary parent's actual mlink.fid for the same filename instead.

* fix: review findings on #29 — substring perm match, test isolation

Blocking: permToMperm switched on the whole perm string, matching only exact "x"/"l". Canonical manifest_file_mperm (src/manifest.c:1482-1492) does a substring test (strstr), and F-card perm fields can carry more than one character — internal/deck/parse.go:194 assigns the field verbatim from remote xfer input, and Fossil itself emits multi-char perm strings (the " w" rename placeholder, see #51). An exact match on e.g. "wx" would silently drop the executable bit, the exact invariant PR #48 landed to protect. Switch to strings.Contains, x checked before l to match canonical's order (the same idiom already used at internal/checkout/vfile.go:52). Added TestPermToMperm_SubstringMatch, including a "wx" case pinned to mperm=1.

Test isolation: rowFor in both mlink_test.go table-driven tests closed over the parent *testing.T, so a Fatalf in one subtest's row lookup aborted every subtest after it via runtime.Goexit unwinding the shared parent goroutine. Pass each subtest's own t into rowFor instead. Proven on the reviewer's revert run: after the deletion subtest's Fatalf, the merge-add and normal-add subtests never even started.

Tightened the carried-over-file assertion in TestInsertMlinks_MergeParentsGetPidNegativeOne from "pid != 0" (would pass for -1 too) to pin the exact parent fid, matching how the crosslink-path twin test already asserts it.

Documented (comment only, no behavior change) a known divergence from canonical in resolveMlinkParent: a file present in a merge parent with different content (a conflict resolved during the merge) should stay pid=0 in canonical Fossil, but this function returns pid=-1 for any file present in a merge parent regardless of content match. Low blast radius — every mlink consumer in this package filters pid != fid, which holds either way — but worth flagging so it isn't rediscovered as a surprise. check-in: 24c35dad87 user: [email protected] tags: trunk

* checkout: Update returns UpdateResult so conflicts are visible

The internal 3-way merge already tracked which files were written, removed, and left with conflict markers, but Checkout.Update discarded all of it and returned a bare error. That collapsed a clean update and one that silently wrote <<<<<<< markers into working-tree files into the same outcome, with no way for a caller to tell them apart.

Update now returns (UpdateResult, error). UpdateResult.Conflicted lists paths that were merged but not cleanly -- this is still a successful update (err == nil), not an error, since the working tree is usable and the caller just needs to know which files to show a user. A genuine failure still returns a non-nil error paired with a zero-value UpdateResult. FilesWritten/FilesRemoved are populated the same way, as paths rather than counts.

Threading the paths up required changing the internal update's processFileUpdates from returning bare int counts to returning the actual []string paths, since the count was all that existed before. Checkout.Extract is untouched -- it has no conflict concept.

Breaking change, clean break at v0.7.0 (pre-1.0).

* checkout: sort Update result paths, document FilesWritten/Conflicted overlap

Update's result slices were built by ranging over maps.allNames, a map[string]bool, so their order was whatever Go's randomized map iteration happened to produce that run. A caller diffing result.FilesWritten against a golden file, or asserting on it with reflect.DeepEqual, would see it fail intermittently for no reason related to their code.

Sort all three slices (sort.Strings) right before Update returns. Added a test that touches four files -- two conflicting, two clean additions, named so map iteration order is very unlikely to hand them back sorted by luck -- and asserts the exact sorted slice on both FilesWritten and Conflicted. Verified it fails reliably (5/5 runs) with the sort calls temporarily removed, then restored the fix.

Also stated outright, in both the internal and public UpdateResult doc comments, what "clean or not" was only implying: every path in Conflicted also appears in FilesWritten, since the conflict markers are what got written there. check-in: 75f3521e54 user: [email protected] tags: trunk

* fix: make Timeline pagination cursor an exact total order

#54 (merged as 95cbbb7) split Repo.Timeline into a repository-wide enumeration and Repo.Ancestry, the original first-parent walk under an honest name. Its pagination cursor predicate used a tolerance window around the caller-supplied Before/After pair to work around a lossy julian<->time.Time round trip:

mtime < (C - eps) OR (mtime <= (C + eps) AND rid < After)

This reintroduces exactly the defect the composite cursor existed to prevent, and makes it worse: rows within eps of the boundary that are not real ties get filtered by rid instead of tie-broken by it, which can skip a row, duplicate another, or fail to advance the cursor at all — unlike canonical fossil's own bare-timestamp slop, which repeats rows but still advances. A pagination test using only exact-tie and one-hour-apart fixtures could not exercise the 0 < delta < eps range where the defect lives, so it stayed green.

Fix: make the predicate an exact total order, with rid as a true tie-break only at exact mtime equality:

mtime < C OR (mtime = C AND rid < After)

and stop deriving C from a time.Time altogether. TimelineOpts.After is now an opaque Cursor (internal/fsltype.Cursor, re-exported as libfossil.Cursor): obtained only from a LogEntry the library already produced, carrying the exact float64 mtime read off that row rather than one reconstructed from a rounded timestamp. A cursor built from parts can silently drift from the row it claims to follow; a cursor that can only ever be handed back cannot.

That exactness depends on the row's scanned mtime being faithful to begin with, which surfaced a second bug: under the ncruces/WASM sqlite driver, event.mtime scans as a time.Time via the driver's own internal REAL conversion, carrying ~13us of noise that db.ScanJulianDay's time.Time branch then truncated (via UnixMilli()) into a full missed millisecond — independent of and in addition to the JulianToTime truncation bug #54 already fixed on the read-back side. Fixed db.timeToJulian to round to the nearest millisecond, the same class of fix, at the point where the driver-specific noise actually enters.

Also tightened the regression coverage: - Pagination tests now use a fixed loop bound of exactly ceil(n/pageSize) pages rather than an open "while non-empty" loop, so a non-advancing cursor fails the test immediately instead of hanging it. - Added a sub-millisecond-boundary fixture (events 3ms apart, well inside the old epsilon window) with rid order deliberately decoupled from mtime order — a monotonic fixture cannot expose this class of bug, since rid and mtime need not move together in general (backdated commits, imports). - TestJulianRoundTrip now asserts exact equality for values originating from TimeToJulian instead of a *1ms tolerance, which would have passed under both the old truncating JulianToTime and the current rounding one and so was not a regression guard for that fix.

No change to Ancestry, event-type filtering, NULL tolerance, or either loop bound; all were already correct.

* fix: assert pagination termination; mark Cursor change breaking

Both pagination tests (the pre-existing ones and the sub-millisecond regression) ran exactly ceil(n/pageSize) pages and asserted each event appeared once, but neither ever called Timeline one more time to confirm the next page comes back empty. The headline symptom of the epsilon-cursor defect these tests exist to catch was non-termination — a caller paginating until empty never stops — and that specific failure mode was going unguarded: a cursor that enumerates every row correctly but never signals exhaustion would have passed all four tests as they stood.

Added one more Timeline call after the last expected page in all four tests, asserting it returns zero entries. Verified this is not vacuous: reverted the tie-break to <= (non-strict) and confirmed the existing per-event count checks catch it; separately, made Timeline phantom-return one row only on an already-exhausted call (isolated from the real per-page counts, which stayed correct in that experiment) and confirmed only the new final-page assertion fails — proving it exercises a real, non-redundant path, not one already covered by the count checks. Both experiments reverted before commit.

Also corrected the CHANGELOG bullet describing the Cursor redesign: it described the new Cursor type but never stated that TimelineOpts.Before (time.Time) and TimelineOpts.After (FslID) were deleted outright, not deprecated. A reader diffing releases would have read it as an addition rather than a second compile-breaking change on top of #54. check-in: 168e20fc91 user: [email protected] tags: trunk

Repo.Timeline was an ancestry walk wearing an enumeration's name: it required a start rid, emitted that check-in, followed plink where isprim=1 to the primary parent, and repeated until the chain ended. It never touched the event table and never visited a second parent or a sibling branch head, so it silently omitted any check-in that wasn't a first-parent ancestor of the given start rid — on libfossil's own repository that was 577 of 3,761 check-ins, with no error and no warning. The limit option never bound because the (wrong) walk exhausted first.

Split into two operations under honest names:

- Repo.Timeline(TimelineOpts) enumerates the event table newest first, every kind by default or a single kind via Type. This matches canonical `fossil timeline`'s -t/--type default (a filter only applies when given and not 'a'). - Repo.Ancestry(LogOpts) keeps the original first-parent walk, unchanged in behavior, under a name that says what it does.

Breaking change, no deprecated shim (pre-1.0, clean break): callers of the old Timeline(LogOpts) need to move to Ancestry(LogOpts) if they want the walk, or to Timeline(TimelineOpts) if they want the enumeration fossil's own timeline command performs. The only in-tree callers were the `repo timeline` CLI command and the two examples; all three are migrated to Timeline(TimelineOpts). See CHANGELOG.md.

LogEntry gains a Kind field identifying which of event.type's six kinds (ci, e, f, g, t, w) an entry is; Parents is only populated for check-ins, since plink only relates check-in artifacts.

Timeline's pagination cursor orders by (mtime DESC, rid DESC) instead of canonical fossil's bare mtime DESC with no tie-break — a deliberate improvement, not an oversight, since fossil's web cursor combines that missing tie-break with a one-second timestamp slop and can repeat or skip rows at a page boundary. Getting the composite cursor to actually round-trip losslessly required fixing a real bug along the way: JulianToTime truncated instead of rounded when converting a julian day back to milliseconds, so TimeToJulian(JulianToTime(m)) was not always equal to m even for exact-millisecond inputs — enough to misplace a row at a page boundary under exactly the same-mtime conditions this issue's acceptance criteria test for. Fixed to round to nearest. Because the same class of drift also appears in the ncruces/WASM sqlite driver's own internal REAL-to-time.Time conversion (independent of this library's code), the cursor comparison also carries a small explicit tolerance rather than relying on bit-exact equality across drivers.

Both the ancestry walk's loop and the enumeration's row loop now carry an explicit, testable bound so a corrupt or cyclic plink table can't hang the process. check-in: 9a2697c556 user: [email protected] tags: trunk

* fix: separate blob existence from blob availability (#43)

A phantom is a real blob row with size = -1 and NULL content, so blob.Exists — "SELECT rid FROM blob WHERE uuid=?" — answers true for it. Every guard built on Exists therefore admitted artifacts whose content cannot be read, and content.Expand aborted with "rid N is a phantom".

Canonical Fossil keeps two predicates and never substitutes one for the other: rid_from_uuid (existence, xfer.c:70) and content_is_available (availability, content.c:163). The latter is transitive — it follows delta.srcid up the chain, so a delta whose base is a phantom is unavailable even though every row in the chain exists.

Add content.IsAvailable / content.AvailableByUUID, ported from content_is_available including the chain walk, with an explicit depth bound so a cyclic delta table terminates instead of hanging. The walk lives in exactly one place; no caller touches the delta table.

blob.Exists keeps its behavior and gains a doc comment stating that it answers existence only, returns true for phantoms, and naming the availability predicate as the alternative.

manifest.blobPresent is deleted. It was a local patch at one call site that tested size >= 0 without the chain walk, so it still reported a delta-on-phantom as present — the same class of bug it was written to work around.

Every blob.Exists call site was classified as existence or availability and converted where it guards a content read. Sites that want rid identity — plink/mlink/tagxref population, cluster phantom creation, private-status bookkeeping, the post-store sanity check in handleFile, and phantom filling in storeReceivedFile — keep existence, since delta-before-base is a normal steady state during clone.

Verified against a loopback fossil 2.28 server: the 23-artifact repro that previously failed roughly half the time now clones 6 of 6.

* fix: IsAvailable must fail closed when the delta lookup errors

Only sql.ErrNoRows means the chain is grounded. Any other failure — cancelled context, closed connection, NULL srcid — says nothing about groundedness, so returning true there claims readable content that was never verified. That is the same class of bug this predicate exists to prevent, and it runs on data arriving over the wire during clone.

The blob-size lookup above already fails closed on the same pattern; this restores the symmetry. check-in: 670ca0e75f user: [email protected] tags: trunk

Manage() inserted new vfile rows with isexe always defaulting to 0, regardless of the file's on-disk permissions. buildCommitFiles() also never consulted vfile.isexe when constructing the manifest F-card for changed/added files — it only carried forward the parent manifest's Perm string, which is empty for a brand new file. Together this meant an executable file added via Manage always committed without an "x" marker, and a later Extract wrote it back as 0644.

Fix: - Manage() now stats the file and records isexe using the same predicate as Fossil's file_perm() (src/file.c:316): owner-execute on a regular file (S_ISREG && S_IXUSR). Group/other-only execute (e.g. 0645) does not count, matching canonical behavior. The detection lives in one place (modeIsExecutable, next to the existing isexe-to-mode helper vfilePerm) rather than being duplicated at call sites. - buildCommitFiles() now derives the F-card Perm for changed/added files from vfile.isexe instead of only inheriting the parent's Perm, without disturbing an existing symlink marker ("l").

Extract already read isexe correctly when writing files back to disk, so no change was needed there.

Out of scope, left untouched: symlink (islink) handling, and executable-bit CHANGE detection on an already-tracked file whose mode changes without going through Manage again (Fossil's belt-and-braces re-stat at commit time / VFileIsExec path) — that path appears to still be missing here and should be tracked separately if it matters. check-in: 76dd5241c2 user: [email protected] tags: trunk

Fossil's check-in manifests require only the D and Z cards; the U-card (user) and C-card (comment) are optional, and canonical fossil binds a missing U-card as SQL NULL on the event row. manifest.Log scanned both columns into plain strings, so any repository containing such a check-in failed the entire walk with "converting NULL to string is unsupported" and returned zero rows.

Scan event.user and event.comment through sql.NullString and use its zero-value String field, so a NULL column becomes "" without changing LogEntry's public shape - no nullable type leaks out, and no caller has to know NULL was ever possible. Non-NULL columns are unaffected.

repo timeline now renders an empty user as "?", matching canonical fossil's coalesce(euser,user,'?') TTY output; the substitution stays in the CLI's presentation layer, not the library.

Fixes #44 check-in: 9564ccad55 user: [email protected] tags: trunk

Newly created repositories were missing the ticket and ticketchng tables and the ticketchng_idx1 index that fossil new provisions. Stock Fossil's web UI queries the ticket table unconditionally (e.g. info.c's artifact-detail lookup: SELECT 1 FROM ticket WHERE tkt_uuid GLOB ...), so opening a libfossil-created repository with the stock fossil binary and visiting /info/<uuid> crashed with "no such table: ticket".

Add the ticket and ticketchng table definitions and the ticketchng_idx1 index to schemaRepo2, taken verbatim from zRepositorySchema2 in fossil's src/schema.c, in the same position canonical fossil places them (after attachment_idx2, before cherrypick).

Verified against a real fossil new repository: sqlite_master table/ index diff and per-table .schema diff for ticket/ticketchng now match exactly. check-in: 099ddb1b7a user: [email protected] tags: trunk

Bumps the go_modules group with 1 update in the / directory: [golang.org/x/crypto](https://github.com/golang/crypto). Bumps the go_modules group with 1 update in the /observer/otel directory: [golang.org/x/crypto](https://github.com/golang/crypto).

Updates `golang.org/x/crypto` from 0.49.0 to 0.52.0 - [Commits](https://github.com/golang/crypto/compare/v0.49.0...v0.52.0)

Updates `golang.org/x/crypto` from 0.49.0 to 0.52.0 - [Commits](https://github.com/golang/crypto/compare/v0.49.0...v0.52.0)

--- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-version: 0.52.0 dependency-type: direct:production dependency-group: go_modules - dependency-name: golang.org/x/crypto dependency-version: 0.52.0 dependency-type: indirect dependency-group: go_modules ...

Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> check-in: c5e189600f user: [email protected] tags: trunk

* docs: rebrand project to go-libfossil; add acknowledgements

Rename the project/repo identity to go-libfossil while keeping the Go module path (github.com/danmestas/libfossil) and the libfossil CLI/package names unchanged — imports and installs are unaffected, GitHub redirects cover the renamed repo URLs.

- README: title, Tests badge repo URL, and a note clarifying the repo-name vs import-path split. - Add an Acknowledgements section crediting Stephan Beal (libfossil), Dr. D. Richard Hipp / Fossil SCM, TigerBeetle TigerStyle, John Ousterhout (A Philosophy of Software Design), and Antithesis (DST), with links. - Docs site: hugo.yaml title + GitHub repo links, landing page identity surfaces. baseURL, module paths, and CLI/API names left intact.

* docs: add go-libfossil logo to README check-in: 97ae7eb00d user: [email protected] tags: trunk