Artifact b0e6ed9840c13da08becad99a97dfd503dd22dd7d48751abbb0a20ba4632d6ed:
- File
internal/manifest/dephantomize_defer_test.go
— part of check-in
[5e6ef35173]
at
2026-07-28 02:35:56
on branch trunk
— fix: derive mlink, plink, tagxref and leaf the way canonical fossil does (#193) (#211)
A full clone of the Fossil self-hosting repository disagreed with what canonical fossil derives from the SAME artifacts: mlink -3,351 rows, tagxref -632, plink -5, leaf -1. `fossil rebuild` on our own clone put every one of them back, so the artifact set was complete and the fault was entirely in crosslink-time derivation.
Six independent causes, each verified against `fossil rebuild` on a 67.6k-blob corpus clone:
mlink, -3,351. An mlink row records a parent-to-child TRANSITION, and a merge check-in has more than one parent. Canonical runs the same file-level diff once per parent -- primary, then each merge parent, then each cherrypick source -- and flags the extra rows isaux=1; we only ever diffed against the primary parent. add_mlink is now ported whole (internal/manifest/addmlink.go), which also brings the two passes the old code approximated: the delta-vs-delta revert/resurrect pass, and the baseline-child deletion pass with its mperm=0 delete rows. "Added by merge" (pid=-1) follows canonical's rule too -- a count over the rows a file produced, not a name lookup in the merge parents. A parent whose manifest cannot be read now yields NO rows rather than a whole tree of bogus "added" rows, and the rows are derived later from the parent's side once it arrives (linkPlinkChildren), so the result no longer depends on the order the sweep visits artifacts in.
tagxref, -612 of -632. A check-in's T-card does not have to name the check-in carrying it: `fossil merge --integrate` records the branch it closed as `T +closed <hash>` on the merge check-in itself. Canonical applies a check-in's, a control artifact's and a technote's T-cards through one code path; we applied only the self-referential ones on check-ins and none at all on technotes.
tagxref, the rest. A wiki-
or event-<id> tag's value is the W-card body measured after LEADING whitespace is skipped, not the raw card length.
Artifact identification. The sweep parses every blob in the repository, so what counts as an artifact decides which rows exist at all. Three gaps: a bare card letter with no arguments is legal (fossil's own first check-in spells its parentless-ness as a bare "P" card) and rejecting it dropped that check-in entirely; an N-card after a P-card is legal in a technote for historical compatibility, and rejecting it dropped real technote revisions with their event-<id> and bgcolor tags; and the check-in-versus-control decision turns on a comment, files or parents, which made `merge --integrate` check-ins read as control artifacts. Fossil's required-card table now gates linking (isArtifact), on both the sweep and the dephantomize cascade, so a file whose first line parses as a Z-card no longer links as a dateless check-in and a wiki revision missing its U-card no longer links at all.
plink, -5. The ancestry a check-in declares is a property of the check-in, not of what happens to be in the repository when it is crosslinked. A parent hash with no blob yet now reserves a phantom the way canonical's uuid_to_rid does, so the edge exists whether the parent arrives in a later round or never; skipping it lost the edge for good, because nothing revisits an already-linked artifact.
leaf, -1. Nothing calls leaf_rebuild() outside `fossil leaves --recompute`; the table is maintained by leaf_check() over the rids leaf_eventually_check() schedules -- each rid a branch tag lands on, plus that rid's parents. Drawing candidates from event instead both missed phantom parents (which fossil does list) and added check-ins that carry no branch tag and no ancestry (which it does not).
After: a fresh corpus clone and `fossil rebuild` of that clone agree row for row on mlink, plink, tagxref, leaf AND event -- 104,919 / 23,741 / 51,766 / 1,426 / 27,099 either way, and `fossil test-integrity` reports 0 errors.
Not addressed here: the local commit path (manifest.insertMlinks) still writes one mlink row per file rather than per change, and does not record merge-parent transitions. That is a separate divergence from the clone parity this issue is about.
Co-authored-by: Dan Mestas <[email protected]> (user: [email protected], size: 3691) [annotate] [blame] [check-ins using]
package manifest import ( "context" "testing" "time" "github.com/danmestas/go-libfossil/internal/blob" "github.com/danmestas/go-libfossil/internal/deck" "github.com/danmestas/go-libfossil/internal/hash" _ "github.com/danmestas/go-libfossil/internal/testdriver" ) // TestAfterDephantomizeCascadeDefersCheckinWithMissingFileBlob is the // cascade's analogue of TestCrosslink_DefersCheckinWithMissingFileBlob // (issue #180): the mid-round phantom-fill cascade must hold back a Checkin // manifest whose referenced blob has not arrived, exactly as the // whole-repository sweep's deferCheckin does, so a later Crosslink sweep -- // triggered once the missing blob arrives -- can still rediscover and // complete it. // // Before the fix, afterDephantomize called linkArtifact for a Checkin // unconditionally: it wrote the event row even though insertCheckinMlinks // silently skipped the missing-blob F-card. Because // collectCrosslinkCandidates only reselects a rid with NO event row, that // half-linked checkin was permanently stranded -- no future sweep, no matter // how many rounds ran or when the blob arrived, would ever revisit it. A // downstream Checkout.Update walking to that checkin would hit "blob not // found" with no path to recover. func TestAfterDephantomizeCascadeDefersCheckinWithMissingFileBlob(t *testing.T) { r := setupTestRepo(t) fileContent := []byte("cascade race: file content arrives later") fileUUID := hash.SHA1(fileContent) // A Checkin manifest referencing fileUUID, stored as full (non-delta) // content -- what storeResolvedContent produces for a "file" card, and // exactly the dephantomizedRid that storeReceivedFile then hands to // manifest.AfterDephantomize. The file blob is deliberately never stored. d := &deck.Deck{ Type: deck.Checkin, C: "cascade race with missing file blob", U: deck.User("tester"), D: time.Date(2026, 7, 27, 12, 0, 0, 0, time.UTC), F: []deck.FileCard{{Name: "race/file.txt", UUID: fileUUID}}, // A root check-in declares its branch -- both fossil's own `init` and // this package's Checkin do -- and leaf only ever considers a check-in // that carries a branch tag. See repairLeafTable. T: []deck.TagCard{ {Type: deck.TagPropagating, Name: "branch", UUID: "*", Value: "trunk"}, {Type: deck.TagSingleton, Name: "sym-trunk", UUID: "*"}, }, } manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } manifestRID, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("blob.Store(manifest): %v", err) } // Drive the manifest through the cascade exactly as // storeReceivedFile -> manifest.AfterDephantomize does for a just-stored // full-content blob. The file blob it references is missing, so this must // defer: no event/leaf/mlink rows at all. AfterDephantomize(context.Background(), r, manifestRID) assertCounts(t, r, manifestRID, 0, 0, 0, "after cascade, blob missing") // The missing file blob arrives (a later round). The next Crosslink sweep // -- HandleSync's post-round trigger, or sync.Clone's terminal sweep -- // must rediscover the manifest (no event row exists yet) and link it // fully. This half is what proves recovery is actually restored: a defer // that never gets picked back up would still pass the assertion above. if _, _, err := blob.Store(r.DB(), fileContent); err != nil { t.Fatalf("blob.Store(file): %v", err) } linked, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink (post-arrival phase): %v", err) } if linked != 1 { t.Errorf("Crosslink (post-arrival phase): linked = %d, want 1", linked) } assertCounts(t, r, manifestRID, 1, 1, 1, "after blob arrival") }