Artifact e0f37b08a1058d2b6075f981bfec34e672752f1f70c24f52c6b38e95378aa57f:
- File
internal/tag/propagate.go
— part of check-in
[1f59012870]
at
2026-07-23 10:17:46
on branch trunk
— manifest: visit Crosslink candidates in delta-chain order (#126)
* manifest: create forumpost on demand, matching canonical rebuild
Crosslink's candidate query names the forumpost table unconditionally, but canonical fossil creates it lazily -- only once a forum artifact needs it -- and fossil rebuild drops it for a repository that never populated it. Crosslink now recreates the table itself before the sweep runs, matching canonical's on-demand behavior instead of erroring with "no such table: forumpost" against a rebuilt repo.
Audited the rest of the on-demand-in-canonical set (ticket, ticketchng) against the current schema: both are created eagerly by canonical at init and survive rebuild, so they don't share this problem.
Added TestCrosslinkForumPostTableDropped (drops the table directly) and TestCrosslinkAfterFossilRebuild (drives an actual fossil rebuild first) to cover the regression, and updated the existing fossilbin comparison test's comment now that the rebuild-last ordering is no longer required.
* manifest: visit crosslink candidates in delta-chain order
Crosslink swept candidates in ascending rid order, but content_deltify stores a blob's older versions as deltas against its newer ones, so delta chains run the opposite direction: high-rid root down to low-rid ancestors. An ascending sweep's first candidate of a chain therefore materializes the whole chain, and the rest are consumed arbitrarily far in the future -- the working set of a whole-repository sweep was the entire repository expanded (8.2 GiB for the Fossil SCM repository).
Candidates are now topologically sorted so a delta's base is always visited before its dependents (deltaChainOrder), matching Fossil's own rebuild_step shape: root first, each descendant one delta application from an already-expanded predecessor. The working set this sweep needs is now bounded by how many chains are in flight at once, not by repository size.
That reordering also changes the order artifact side effects get written in, since content expansion and artifact crosslinking share one loop. Most of what Crosslink writes turned out to already be a pure function of its own artifact (event/plink/mlink/tagxref-origin rows) and needed no change. Two things generalize across the whole plink graph rather than any one artifact and were genuinely order-sensitive:
- leaf table bookkeeping, which inserted the new checkin and deleted its parent -- correct only when parents are linked before their children, which delta-chain order no longer guarantees. - tag propagation, which ran once per checkin from its immediate parent -- reachable only to children plink already held at that moment.
Both are now full, order-independent repair passes (repairLeafTable, repairTagPropagation) run once after the sweep, mirroring Fossil's own manifest_crosslink_end + leaf_rebuild() shape: defer the order-sensitive work and repair it once rather than trying to preserve order through the sweep. tag.propagate's mtime-gated, plink-live-at-call-time design turned out to already be confluent -- replaying every self-declared tag origin once, in any order, converges on the same tagxref state -- so the repair is just doing that instead of depending on visiting order to get there implicitly. Tech-note event replacement and forum thread titles turned out to already have the same confluent shape and needed no change; left comments explaining why.
Also fixes a tag.PropagateAll crash on any origin with a NULL tagxref value (e.g. CrosslinkCluster's cluster tag), surfaced by the new propagation repair actually reaching those rows for the first time.
Regression coverage: a real three-generation checkin chain, forced into a genuine manifest delta chain, that would have left two of three checkins untagged and all three marked leaf under the old per-checkin propagation and incremental leaf bookkeeping.
* fix: address review nits on crosslink order-sensitivity and delta cycles
- deltaChainOrder now returns an error instead of panicking when the candidate delta graph fails to drain. The delta table is on-disk data that can arrive over sync from an untrusted peer, so a cycle (corrupt or hostile repo) should surface as an error rather than crash the process.
- snapshotDerived's digest now covers tagxref (rid/tagid/tagtype/ srcid/origid/value), and TestFossilBinaryReadsCrosslinkedRepo compares it against canonical fossil rebuild -- closing the gap the test previously called out by name ("tagxref is the notable gap").
- Added TestFossilBinaryReadsCrosslinkedRepoBranchAndMerge, exercising the order-sensitive paths incrementalHistory's single-branch history cannot reach: a feature branch diverging from trunk, and a merge commit whose own branch tag must come from its primary parent, not the merged-in one. event/plink/tagxref match canonical fossil row-for-row on this topology.
leaf and mlink are deliberately excluded from that comparison: the fixture surfaced that both have pre-existing gaps against canonical fossil (repairLeafTable counts any plink edge rather than only primary-parent edges; insertCheckinMlinks writes a row for every F-card rather than only changed ones) that predate this fix and are out of scope for #102/#103.
(user: [email protected], size: 7192) [annotate] [blame] [check-ins using]
package tag import ( "container/heap" "database/sql" "fmt" "github.com/danmestas/go-libfossil/db" libfossil "github.com/danmestas/go-libfossil/internal/fsltype" ) // propagate walks the plink DAG from a target artifact to all descendants, // inserting/deleting tagxref rows to cascade propagating tags. // This matches Fossil's tag_propagate in tag.c:34-113. // // For propagating tags (type 2), it inserts tagxref rows with srcid=0. // For cancel tags (type 0), it deletes all tagxref entries for that tag. // Special case: if tagName is "bgcolor", also updates event.bgcolor. func propagate(q db.Querier, tagid int64, tagType int, origID libfossil.FslID, mtime float64, value string, tagName string, pid libfossil.FslID) error { if q == nil { panic("tag.propagate: q must not be nil") } // Priority queue seeded with the target artifact (mtime=0.0 for seed) pq := &mtimeQueue{} heap.Init(pq) heap.Push(pq, &queueItem{rid: pid, mtime: 0.0}) // Guard against DAG cycles or pathologically deep chains. // Fossil repos typically have <100K checkins; 1M is a safe upper bound. const maxIterations = 1_000_000 // Visited set prevents re-processing nodes reachable via multiple paths. visited := make(map[libfossil.FslID]bool) iterations := 0 for pq.Len() > 0 { iterations++ if iterations > maxIterations { return fmt.Errorf("tag.propagate: exceeded %d iterations (possible cycle)", maxIterations) } item := heap.Pop(pq).(*queueItem) currentRid := item.rid if visited[currentRid] { continue } visited[currentRid] = true // Query primary children via LEFT JOIN with tagxref. // The doit column is 1 if we should propagate/cancel to this child. // For propagating: doit = 1 if (srcid=0 AND tagxref.mtime < ?) OR no tagxref entry // For cancel: doit = 1 if there's any tagxref entry with srcid=0 (propagated tag) var query string if tagType == TagPropagating { query = ` SELECT cid, plink.mtime, COALESCE(srcid=0 AND tagxref.mtime < ?, 1) AS doit FROM plink LEFT JOIN tagxref ON cid=tagxref.rid AND tagxref.tagid=? WHERE pid=? AND isprim=1 ` } else { // For cancel, we want to delete any propagated tag (srcid=0) query = ` SELECT cid, plink.mtime, COALESCE(srcid=0, 0) AS doit FROM plink LEFT JOIN tagxref ON cid=tagxref.rid AND tagxref.tagid=? WHERE pid=? AND isprim=1 ` } var rows *sql.Rows var err error if tagType == TagPropagating { rows, err = q.Query(query, mtime, tagid, currentRid) } else { rows, err = q.Query(query, tagid, currentRid) } if err != nil { return fmt.Errorf("query children of %d: %w", currentRid, err) } children := []struct { cid libfossil.FslID childMtime float64 doit bool }{} for rows.Next() { var cid int64 var childMtimeRaw any var doitInt int if err := rows.Scan(&cid, &childMtimeRaw, &doitInt); err != nil { rows.Close() return fmt.Errorf("scan child: %w", err) } childMtime, _ := db.ScanJulianDay(childMtimeRaw) children = append(children, struct { cid libfossil.FslID childMtime float64 doit bool }{libfossil.FslID(cid), childMtime, doitInt != 0}) } rows.Close() if err := rows.Err(); err != nil { return fmt.Errorf("rows error: %w", err) } // Process each child for _, child := range children { if child.doit { if tagType == TagPropagating { // Insert propagating tagxref entry if _, err := q.Exec( `REPLACE INTO tagxref(tagid, tagtype, srcid, origid, value, mtime, rid) VALUES(?, 2, 0, ?, ?, ?, ?)`, tagid, origID, value, mtime, child.cid, ); err != nil { return fmt.Errorf("propagate to %d: %w", child.cid, err) } // Special case: bgcolor updates event table if tagName == "bgcolor" { if _, err := q.Exec("UPDATE event SET bgcolor=? WHERE objid=?", value, child.cid); err != nil { return fmt.Errorf("update event bgcolor for %d: %w", child.cid, err) } } } else { // Cancel: delete all tagxref entries for this tag and rid if _, err := q.Exec("DELETE FROM tagxref WHERE tagid=? AND rid=?", tagid, child.cid); err != nil { return fmt.Errorf("cancel at %d: %w", child.cid, err) } // Special case: bgcolor cancellation clears event.bgcolor if tagName == "bgcolor" { if _, err := q.Exec("UPDATE event SET bgcolor=NULL WHERE objid=?", child.cid); err != nil { return fmt.Errorf("clear event bgcolor for %d: %w", child.cid, err) } } } // Queue child for further processing heap.Push(pq, &queueItem{rid: child.cid, mtime: child.childMtime}) } } } return nil } // queueItem represents a node in the priority queue. type queueItem struct { rid libfossil.FslID mtime float64 index int // heap index } // mtimeQueue implements heap.Interface for mtime-ordered priority queue. type mtimeQueue []*queueItem func (pq mtimeQueue) Len() int { return len(pq) } func (pq mtimeQueue) Less(i, j int) bool { return pq[i].mtime < pq[j].mtime } func (pq mtimeQueue) Swap(i, j int) { pq[i], pq[j] = pq[j], pq[i] pq[i].index = i pq[j].index = j } func (pq *mtimeQueue) Push(x any) { n := len(*pq) item := x.(*queueItem) item.index = n *pq = append(*pq, item) } func (pq *mtimeQueue) Pop() any { old := *pq n := len(old) item := old[n-1] old[n-1] = nil item.index = -1 *pq = old[0 : n-1] return item } // PropagateAll re-propagates all tags from rid to its descendants. // Matches Fossil's tag_propagate_all (tag.c:118-135). // Singleton tags (type 1) are treated as cancel (type 0) during propagation. func PropagateAll(q db.Querier, rid libfossil.FslID) error { if q == nil { panic("tag.PropagateAll: q must not be nil") } if rid <= 0 { return nil } rows, err := q.Query( "SELECT tagid, tagtype, mtime, value, origid FROM tagxref WHERE rid=?", rid, ) if err != nil { return fmt.Errorf("tag.PropagateAll query: %w", err) } defer rows.Close() type entry struct { tagid int64 tagtype int mtime float64 value string origid libfossil.FslID } var entries []entry for rows.Next() { var e entry var mtimeRaw any var value sql.NullString // tagxref.value is nullable (e.g. CrosslinkCluster's cluster tag) if err := rows.Scan(&e.tagid, &e.tagtype, &mtimeRaw, &value, &e.origid); err != nil { return fmt.Errorf("tag.PropagateAll scan: %w", err) } e.value = value.String e.mtime, _ = db.ScanJulianDay(mtimeRaw) entries = append(entries, e) } if err := rows.Err(); err != nil { return fmt.Errorf("tag.PropagateAll rows: %w", err) } for _, e := range entries { tagtype := e.tagtype if tagtype == TagSingleton { tagtype = TagCancel // Matches Fossil: if(tagtype==1) tagtype=0; } var tagname string if err := q.QueryRow("SELECT tagname FROM tag WHERE tagid=?", e.tagid).Scan(&tagname); err != nil { return fmt.Errorf("tag.PropagateAll tagname: %w", err) } if err := propagate(q, e.tagid, tagtype, e.origid, e.mtime, e.value, tagname, rid); err != nil { return fmt.Errorf("tag.PropagateAll propagate tagid=%d: %w", e.tagid, err) } } return nil }