Artifact 4129336491e2612eb153b090eb6c418697b6c9e8ccd58643af2cb0c8e99dcb16:
- File
internal/manifest/leaf.go
— part of check-in
[5e6ef35173]
at
2026-07-28 02:35:56
on branch trunk
— fix: derive mlink, plink, tagxref and leaf the way canonical fossil does (#193) (#211)
A full clone of the Fossil self-hosting repository disagreed with what canonical fossil derives from the SAME artifacts: mlink -3,351 rows, tagxref -632, plink -5, leaf -1. `fossil rebuild` on our own clone put every one of them back, so the artifact set was complete and the fault was entirely in crosslink-time derivation.
Six independent causes, each verified against `fossil rebuild` on a 67.6k-blob corpus clone:
mlink, -3,351. An mlink row records a parent-to-child TRANSITION, and a merge check-in has more than one parent. Canonical runs the same file-level diff once per parent -- primary, then each merge parent, then each cherrypick source -- and flags the extra rows isaux=1; we only ever diffed against the primary parent. add_mlink is now ported whole (internal/manifest/addmlink.go), which also brings the two passes the old code approximated: the delta-vs-delta revert/resurrect pass, and the baseline-child deletion pass with its mperm=0 delete rows. "Added by merge" (pid=-1) follows canonical's rule too -- a count over the rows a file produced, not a name lookup in the merge parents. A parent whose manifest cannot be read now yields NO rows rather than a whole tree of bogus "added" rows, and the rows are derived later from the parent's side once it arrives (linkPlinkChildren), so the result no longer depends on the order the sweep visits artifacts in.
tagxref, -612 of -632. A check-in's T-card does not have to name the check-in carrying it: `fossil merge --integrate` records the branch it closed as `T +closed <hash>` on the merge check-in itself. Canonical applies a check-in's, a control artifact's and a technote's T-cards through one code path; we applied only the self-referential ones on check-ins and none at all on technotes.
tagxref, the rest. A wiki-
or event-<id> tag's value is the W-card body measured after LEADING whitespace is skipped, not the raw card length.
Artifact identification. The sweep parses every blob in the repository, so what counts as an artifact decides which rows exist at all. Three gaps: a bare card letter with no arguments is legal (fossil's own first check-in spells its parentless-ness as a bare "P" card) and rejecting it dropped that check-in entirely; an N-card after a P-card is legal in a technote for historical compatibility, and rejecting it dropped real technote revisions with their event-<id> and bgcolor tags; and the check-in-versus-control decision turns on a comment, files or parents, which made `merge --integrate` check-ins read as control artifacts. Fossil's required-card table now gates linking (isArtifact), on both the sweep and the dephantomize cascade, so a file whose first line parses as a Z-card no longer links as a dateless check-in and a wiki revision missing its U-card no longer links at all.
plink, -5. The ancestry a check-in declares is a property of the check-in, not of what happens to be in the repository when it is crosslinked. A parent hash with no blob yet now reserves a phantom the way canonical's uuid_to_rid does, so the edge exists whether the parent arrives in a later round or never; skipping it lost the edge for good, because nothing revisits an already-linked artifact.
leaf, -1. Nothing calls leaf_rebuild() outside `fossil leaves --recompute`; the table is maintained by leaf_check() over the rids leaf_eventually_check() schedules -- each rid a branch tag lands on, plus that rid's parents. Drawing candidates from event instead both missed phantom parents (which fossil does list) and added check-ins that carry no branch tag and no ancestry (which it does not).
After: a fresh corpus clone and `fossil rebuild` of that clone agree row for row on mlink, plink, tagxref, leaf AND event -- 104,919 / 23,741 / 51,766 / 1,426 / 27,099 either way, and `fossil test-integrity` reports 0 errors.
Not addressed here: the local commit path (manifest.insertMlinks) still writes one mlink row per file rather than per change, and does not record merge-parent transitions. That is a separate divergence from the clone parity this issue is about.
Co-authored-by: Dan Mestas <[email protected]> (user: [email protected], size: 5604) [annotate] [blame] [check-ins using]
package manifest import ( "database/sql" "errors" "fmt" "github.com/danmestas/go-libfossil/db" libfossil "github.com/danmestas/go-libfossil/internal/fsltype" ) // Leaf maintenance, in one place so the sweep and the local commit path cannot // drift apart on what a leaf is. // // Canonical fossil's rule (src/leaf.c, leaf_check/is_a_leaf/leaf_rebuild): a // check-in stops being a leaf only once a child of it carries the *same* branch // name, where a check-in with no branch tag reads as "trunk": // // SELECT 1 FROM plink // WHERE pid=:rid // AND coalesce((SELECT value FROM tagxref // WHERE tagid=TAG_BRANCH AND rid=plink.pid),'trunk') // == coalesce((SELECT value FROM tagxref // WHERE tagid=TAG_BRANCH AND rid=plink.cid),'trunk') // // Two consequences are easy to get wrong and are what issue #189 was: // // - The merge parent of a check-in on another branch stays a leaf. A branch // merged back into trunk keeps its tip in the leaf table -- that class was // 939 of the 1,426 leaves the real Fossil repository has. // - The plink row's isprim flag is not consulted at all. A branch point stays // a leaf even though its only child is its *primary* child, because that // child is on a different branch. // // The branch comparison reads tagxref, so both entry points must run after the // branch tag of every check-in involved is in tagxref -- including the ones a // check-in inherits by propagation rather than declaring itself. // // branchTagID is the tagid db/schema.go seeds for 'branch', which is fossil's // own TAG_BRANCH; both schemas fix it, so it is a constant here rather than a // lookup per row of a whole-repository recompute. const branchTagID = 8 // leafCheck brings rid's leaf row in line with the rule above, inserting or // deleting as needed. It is fossil's leaf_check(), and like fossil's it is // called on a check-in and on each of its parents once the surrounding write is // otherwise complete (leaf_eventually_check + leaf_do_pending_checks). func leafCheck(q db.Querier, rid libfossil.FslID) error { if q == nil { panic("manifest.leafCheck: q must not be nil") } if rid <= 0 { panic("manifest.leafCheck: rid must be positive") } var one int err := q.QueryRow(fmt.Sprintf(` SELECT 1 FROM plink WHERE plink.pid = ? AND coalesce((SELECT value FROM tagxref WHERE tagid=%d AND rid=plink.pid),'trunk') = coalesce((SELECT value FROM tagxref WHERE tagid=%d AND rid=plink.cid),'trunk') LIMIT 1`, branchTagID, branchTagID), rid).Scan(&one) switch { case err == nil: if _, err := q.Exec("DELETE FROM leaf WHERE rid=?", rid); err != nil { return fmt.Errorf("leafCheck delete rid=%d: %w", rid, err) } return nil case errors.Is(err, sql.ErrNoRows): if _, err := q.Exec("INSERT OR IGNORE INTO leaf(rid) VALUES(?)", rid); err != nil { return fmt.Errorf("leafCheck insert rid=%d: %w", rid, err) } return nil default: return fmt.Errorf("leafCheck query rid=%d: %w", rid, err) } } // repairLeafTable recomputes leaf from scratch. Crosslink does not maintain // leaf incrementally as each checkin is linked -- that only produces the right // answer when parents are always linked before their children, which // delta-chain order does not guarantee -- so the whole table is rebuilt once // the sweep's plink edges and propagated branch tags are all in place. // // Which rows fossil's leaf table can contain is decided by which rids fossil // ever runs the rule against, and that is NOT "every check-in". Nothing calls // leaf_rebuild() outside `fossil leaves --recompute`; the table is maintained // by leaf_check(), scheduled by leaf_eventually_check() for a rid and EACH OF // ITS PARENTS whenever a branch tag lands on that rid -- from tag_insert() when // a check-in declares one, and from tag_propagate() when one is inherited // (src/leaf.c, src/tag.c). So the candidate set is: // // every rid carrying a branch tagxref row, plus every rid that is a parent // in plink // // and it differs from "every check-in" in both directions. A parent whose own // artifact never arrived has no event row at all, yet fossil does list such a // phantom as a leaf when the check-in naming it sits on another branch. In the // other direction, a check-in that neither declares nor inherits a branch tag // and has no ancestry -- `fossil merge --integrate` writes one of these, a // comment and a `T +closed` card and nothing else -- is never checked, and is // absent from fossil's leaf table even though the rule would admit it. Drawing // candidates from event instead put four such rows in a corpus clone that // `fossil rebuild` then removed (issue #193). func repairLeafTable(q db.Querier) error { if q == nil { panic("manifest.repairLeafTable: q must not be nil") } if _, err := q.Exec("DELETE FROM leaf"); err != nil { return fmt.Errorf("repairLeafTable clear: %w", err) } if _, err := q.Exec(fmt.Sprintf(` INSERT INTO leaf(rid) SELECT c.rid FROM ( SELECT rid FROM tagxref WHERE tagid=%d UNION SELECT pid AS rid FROM plink WHERE pid > 0 ) c WHERE NOT EXISTS( SELECT 1 FROM plink WHERE plink.pid = c.rid AND coalesce((SELECT value FROM tagxref WHERE tagid=%d AND rid=plink.pid),'trunk') = coalesce((SELECT value FROM tagxref WHERE tagid=%d AND rid=plink.cid),'trunk')) `, branchTagID, branchTagID, branchTagID)); err != nil { return fmt.Errorf("repairLeafTable insert: %w", err) } return nil }