Skip to content
go-libfossil Timeline
Login

Timeline

1 check-in tagged with "v0.4.2"

2026-04-26
06:51
v0.4.2: defer manifest crosslink until referenced blobs arrive (#5)

* sync: defer manifest crosslink until referenced blobs arrive

Repo.HandleSync (added crosslink in v0.4.1 PR #4 fix 3) crosslinked each manifest as soon as its blob arrived, even when the manifest referenced blobs that hadn't yet been received in the same session. A receiver that Pulled during a multi-blob delivery saw the manifest in event/leaf/mlink while one or more file blobs were still in flight, and a downstream Checkout.Update walking those manifests via manifest.ListFiles hit 'expandUUID: blob not found for uuid <hex>'.

Surfaced by agent-infra's hub-and-leaf scaling trial under 16-way concurrent fork+merge cycles (agent-infra trial #10, see docs/trials/2026-04-25/trial-report.md). The 3-leaf friendly case in agent-infra's hub-leaf-e2e doesn't exhibit it because the broadcast cadence is slower than the multi-blob delivery window.

Fix: manifest.Crosslink now defers a Checkin candidate when any referenced blob (F-card files or B-card baseline for delta manifests) is not present locally. The deferral is implicit: the candidate query selects rids without an event/tagxref/forumpost/attachment row, so a deferred manifest re-appears in the next sweep automatically. Since HandleSync runs Crosslink whenever filesRecvd > 0, the next sync round that delivers the missing blob also re-runs the sweep and links the manifest fully.

Divergence from fossil-scm/c: fossil's reference handles this via 'rcvfrom' tracking and ipctest-style deferred crosslink flushes; the Go port reuses the existing whole-repo sweep semantics by checking blob presence at sweep time. Documented inline in manifest.Crosslink and missingCheckinRefs.

Test coverage: - TestCrosslink_DefersCheckinWithMissingFileBlob — manifest with missing F-card blob is deferred (no event/leaf/mlink rows), then fully linked once the file blob arrives. - TestCrosslink_DefersDeltaCheckinWithMissingBaseline — delta manifest with missing B-card baseline is deferred; baseline + delta both link in the next sweep once the baseline blob arrives. - TestCrosslink_LinksWhenAllBlobsPresent — confirms the unchanged synchronous path (all blobs present before Crosslink runs).

All three tests fail without the fix (verified by reverting the crosslink.go change locally) and pass with it.

* manifest: assert non-nil repo and non-empty uuid in blobPresent

Tiger Style: panic at the boundary on programmer errors. Calling blobPresent with a nil receiver or empty uuid is always a bug — empty uuid would silently return false (no row matches the empty-string predicate), masking the upstream caller that lost track of which uuid it intended to check.

The peer helpers Crosslink and missingCheckinRefs already panic on nil inputs; blobPresent was the gap.

* manifest: order Crosslink candidate sweep by rid

Without an explicit ORDER BY, SQLite is free to return candidate rows in any order convenient to its query planner. The deferred-crosslink sweep re-discovers the same rids across multiple sync rounds, so any implicit ordering would let blob-arrival timing leak into observable behavior: per-defer slog streams, pending-item processing order, and any future caller that consumes Crosslink's side effects in sequence.

ORDER BY b.rid pins the sweep to insertion order — the same order blobs were stored — which is the natural arrival order for a single sync and a stable canonical order across replays.

* manifest: emit deferred rids and missing-blob list in Crosslink rollup

The session-end Info log previously reported only counts. When a sync ends with deferred manifests, an operator needs to know which manifests are stuck and which UUIDs they are waiting on — otherwise a permanently-stuck deferral (blob never arrives) is invisible until someone diagnoses it from the per-defer Debug stream.

Add deferred_rids and missing_blobs fields. Sort missing_blobs so the rollup is byte-identical across runs that defer the same set, regardless of map iteration order.

* manifest: cover P-card non-deferral and shared-blob defer cases

The original three defer tests covered F-card and B-card defer plus the all-present happy path. Three gaps remained:

* Multi-manifest sharing a missing blob: prove both manifests defer together and both link when the shared blob lands. Without this, a regression that links one but not the other would slip past.
* Missing-parent (P-card) intentionally does NOT defer: the deferral check is scoped to F-cards and B-cards because P-card targets do not break Checkout.Update. Deferring on missing parents would deadlock partial clones with pruned ancestors. Pin the contract with a regression test.
* F-card duplicate-UUID dedup: the seen-map in missingCheckinRefs already deduplicates, but the behaviour was untested. Two F-cards with different filenames pointing at the same blob must defer once and link to two mlink rows on arrival.
check-in: b6b6c40218 user: [email protected] tags: trunk, v0.4.2