Artifact cbb6dfc5dc4b215deb348752d5a58eb8ea5a6633bf45920a4b564b23e9d866ef:
- File
internal/manifest/manifest_test.go
— part of check-in
[1f59012870]
at
2026-07-23 10:17:46
on branch trunk
— manifest: visit Crosslink candidates in delta-chain order (#126)
* manifest: create forumpost on demand, matching canonical rebuild
Crosslink's candidate query names the forumpost table unconditionally, but canonical fossil creates it lazily -- only once a forum artifact needs it -- and fossil rebuild drops it for a repository that never populated it. Crosslink now recreates the table itself before the sweep runs, matching canonical's on-demand behavior instead of erroring with "no such table: forumpost" against a rebuilt repo.
Audited the rest of the on-demand-in-canonical set (ticket, ticketchng) against the current schema: both are created eagerly by canonical at init and survive rebuild, so they don't share this problem.
Added TestCrosslinkForumPostTableDropped (drops the table directly) and TestCrosslinkAfterFossilRebuild (drives an actual fossil rebuild first) to cover the regression, and updated the existing fossilbin comparison test's comment now that the rebuild-last ordering is no longer required.
* manifest: visit crosslink candidates in delta-chain order
Crosslink swept candidates in ascending rid order, but content_deltify stores a blob's older versions as deltas against its newer ones, so delta chains run the opposite direction: high-rid root down to low-rid ancestors. An ascending sweep's first candidate of a chain therefore materializes the whole chain, and the rest are consumed arbitrarily far in the future -- the working set of a whole-repository sweep was the entire repository expanded (8.2 GiB for the Fossil SCM repository).
Candidates are now topologically sorted so a delta's base is always visited before its dependents (deltaChainOrder), matching Fossil's own rebuild_step shape: root first, each descendant one delta application from an already-expanded predecessor. The working set this sweep needs is now bounded by how many chains are in flight at once, not by repository size.
That reordering also changes the order artifact side effects get written in, since content expansion and artifact crosslinking share one loop. Most of what Crosslink writes turned out to already be a pure function of its own artifact (event/plink/mlink/tagxref-origin rows) and needed no change. Two things generalize across the whole plink graph rather than any one artifact and were genuinely order-sensitive:
- leaf table bookkeeping, which inserted the new checkin and deleted its parent -- correct only when parents are linked before their children, which delta-chain order no longer guarantees. - tag propagation, which ran once per checkin from its immediate parent -- reachable only to children plink already held at that moment.
Both are now full, order-independent repair passes (repairLeafTable, repairTagPropagation) run once after the sweep, mirroring Fossil's own manifest_crosslink_end + leaf_rebuild() shape: defer the order-sensitive work and repair it once rather than trying to preserve order through the sweep. tag.propagate's mtime-gated, plink-live-at-call-time design turned out to already be confluent -- replaying every self-declared tag origin once, in any order, converges on the same tagxref state -- so the repair is just doing that instead of depending on visiting order to get there implicitly. Tech-note event replacement and forum thread titles turned out to already have the same confluent shape and needed no change; left comments explaining why.
Also fixes a tag.PropagateAll crash on any origin with a NULL tagxref value (e.g. CrosslinkCluster's cluster tag), surfaced by the new propagation repair actually reaching those rows for the first time.
Regression coverage: a real three-generation checkin chain, forced into a genuine manifest delta chain, that would have left two of three checkins untagged and all three marked leaf under the old per-checkin propagation and incremental leaf bookkeeping.
* fix: address review nits on crosslink order-sensitivity and delta cycles
- deltaChainOrder now returns an error instead of panicking when the candidate delta graph fails to drain. The delta table is on-disk data that can arrive over sync from an untrusted peer, so a cycle (corrupt or hostile repo) should surface as an error rather than crash the process.
- snapshotDerived's digest now covers tagxref (rid/tagid/tagtype/ srcid/origid/value), and TestFossilBinaryReadsCrosslinkedRepo compares it against canonical fossil rebuild -- closing the gap the test previously called out by name ("tagxref is the notable gap").
- Added TestFossilBinaryReadsCrosslinkedRepoBranchAndMerge, exercising the order-sensitive paths incrementalHistory's single-branch history cannot reach: a feature branch diverging from trunk, and a merge commit whose own branch tag must come from its primary parent, not the merged-in one. event/plink/tagxref match canonical fossil row-for-row on this topology.
leaf and mlink are deliberately excluded from that comparison: the fixture surfaced that both have pre-existing gaps against canonical fossil (repairLeafTable counts any plink edge rather than only primary-parent edges; insertCheckinMlinks writes a row for every F-card rather than only changed ones) that predate this fix and are out of scope for #102/#103.
(user: [email protected], size: 39321) [annotate] [blame] [check-ins using]
package manifest import ( "bytes" "fmt" "path/filepath" "testing" "time" "github.com/danmestas/go-libfossil/db" "github.com/danmestas/go-libfossil/internal/blob" "github.com/danmestas/go-libfossil/internal/content" "github.com/danmestas/go-libfossil/internal/deck" "github.com/danmestas/go-libfossil/internal/delta" libfossil "github.com/danmestas/go-libfossil/internal/fsltype" "github.com/danmestas/go-libfossil/internal/repo" "github.com/danmestas/go-libfossil/internal/tag" _ "github.com/danmestas/go-libfossil/internal/testdriver" "github.com/danmestas/go-libfossil/simio" "github.com/danmestas/go-libfossil/testutil" ) func setupTestRepo(t *testing.T) *repo.Repo { t.Helper() path := filepath.Join(t.TempDir(), "test.fossil") r, err := repo.Create(path, "testuser", simio.CryptoRand{}, "") if err != nil { t.Fatalf("repo.Create: %v", err) } t.Cleanup(func() { r.Close() }) return r } func TestCheckinBasic(t *testing.T) { r := setupTestRepo(t) rid, uuid, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "hello.txt", Content: []byte("hello world")}}, Comment: "initial commit", User: "testuser", Time: time.Date(2024, 1, 15, 10, 30, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin: %v", err) } if rid <= 0 { t.Fatalf("rid = %d", rid) } if len(uuid) != 40 && len(uuid) != 64 { t.Fatalf("uuid len = %d", len(uuid)) } var comment string r.DB().QueryRow("SELECT comment FROM event WHERE objid=?", rid).Scan(&comment) if comment != "initial commit" { t.Fatalf("event comment = %q", comment) } var leafCount int r.DB().QueryRow("SELECT count(*) FROM leaf WHERE rid=?", rid).Scan(&leafCount) if leafCount != 1 { t.Fatalf("leaf count = %d", leafCount) } } func TestCheckinFossilRebuild(t *testing.T) { testutil.RequireFossilBin(t) r := setupTestRepo(t) _, _, err := Checkin(r, CheckinOpts{ Files: []File{ {Name: "hello.txt", Content: []byte("hello world")}, {Name: "src/main.go", Content: []byte("package main\n")}, }, Comment: "initial commit", User: "testuser", Time: time.Date(2024, 1, 15, 10, 30, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin: %v", err) } r.Close() if err := testutil.FossilRebuild(r.Path()); err != nil { t.Fatalf("fossil rebuild: %v", err) } } func TestCheckinMultiple(t *testing.T) { r := setupTestRepo(t) rid1, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "file.txt", Content: []byte("v1")}}, Comment: "first", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) rid2, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "file.txt", Content: []byte("v2")}}, Comment: "second", User: "testuser", Parent: rid1, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) var leafCount int r.DB().QueryRow("SELECT count(*) FROM leaf WHERE rid=?", rid1).Scan(&leafCount) if leafCount != 0 { t.Fatal("rid1 still a leaf") } r.DB().QueryRow("SELECT count(*) FROM leaf WHERE rid=?", rid2).Scan(&leafCount) if leafCount != 1 { t.Fatal("rid2 not a leaf") } var plinkCount int r.DB().QueryRow("SELECT count(*) FROM plink WHERE pid=? AND cid=?", rid1, rid2).Scan(&plinkCount) if plinkCount != 1 { t.Fatal("plink missing") } } func TestGetManifest(t *testing.T) { r := setupTestRepo(t) rid, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "hello.txt", Content: []byte("hello")}}, Comment: "test commit", User: "testuser", Time: time.Date(2024, 1, 15, 10, 30, 0, 0, time.UTC), }) d, err := GetManifest(r, rid) if err != nil { t.Fatalf("GetManifest: %v", err) } if d.C != "test commit" { t.Fatalf("C = %q", d.C) } if d.Type != deck.Checkin { t.Fatalf("Type = %d", d.Type) } if len(d.F) != 1 || d.F[0].Name != "hello.txt" { t.Fatalf("F = %+v", d.F) } } func TestListFilesBaseline(t *testing.T) { r := setupTestRepo(t) rid, _, _ := Checkin(r, CheckinOpts{ Files: []File{ {Name: "a.txt", Content: []byte("aaa")}, {Name: "b.txt", Content: []byte("bbb")}, }, Comment: "initial", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) files, err := ListFiles(r, rid) if err != nil { t.Fatalf("ListFiles: %v", err) } if len(files) != 2 { t.Fatalf("count = %d", len(files)) } } func TestLogMultiple(t *testing.T) { r := setupTestRepo(t) rid1, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v1")}}, Comment: "first", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) rid2, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v2")}}, Comment: "second", User: "testuser", Parent: rid1, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) rid3, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v3")}}, Comment: "third", User: "testuser", Parent: rid2, Time: time.Date(2024, 1, 15, 12, 0, 0, 0, time.UTC), }) entries, err := Log(r, LogOpts{Start: rid3}) if err != nil { t.Fatalf("Log: %v", err) } if len(entries) != 3 { t.Fatalf("count = %d", len(entries)) } if entries[0].Comment != "third" || entries[2].Comment != "first" { t.Fatalf("order: %q %q %q", entries[0].Comment, entries[1].Comment, entries[2].Comment) } } func TestLogWithLimit(t *testing.T) { r := setupTestRepo(t) rid1, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v1")}}, Comment: "first", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) rid2, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v2")}}, Comment: "second", User: "testuser", Parent: rid1, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) entries, _ := Log(r, LogOpts{Start: rid2, Limit: 1}) if len(entries) != 1 { t.Fatalf("count = %d", len(entries)) } } // TestLogNullUser covers fossil's optional U-card: a check-in manifest with // no recorded user stores event.user as SQL NULL (the required cards for a // check-in are only "DZ" — see manifest.c:130). Log must tolerate this and // report an empty string, not fail the whole walk. func TestLogNullUser(t *testing.T) { r := setupTestRepo(t) rid1, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v1")}}, Comment: "first", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) rid2, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v2")}}, Comment: "second", User: "testuser", Parent: rid1, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) if _, err := r.DB().Exec("UPDATE event SET user=NULL WHERE objid=?", rid2); err != nil { t.Fatalf("UPDATE event SET user=NULL: %v", err) } entries, err := Log(r, LogOpts{Start: rid2}) if err != nil { t.Fatalf("Log: %v", err) } if len(entries) != 2 { t.Fatalf("count = %d, want 2", len(entries)) } if entries[0].User != "" { t.Fatalf("entries[0].User = %q, want empty string for NULL event.user", entries[0].User) } if entries[0].Comment != "second" { t.Fatalf("entries[0].Comment = %q, want %q", entries[0].Comment, "second") } if entries[1].User != "testuser" { t.Fatalf("entries[1].User = %q, want %q (non-NULL users unchanged)", entries[1].User, "testuser") } } // TestLogNullComment covers fossil's optional C-card, the other realistic // NULL candidate on the same event row (required cards remain only "DZ"). func TestLogNullComment(t *testing.T) { r := setupTestRepo(t) rid1, _, _ := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v1")}}, Comment: "first", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) if _, err := r.DB().Exec("UPDATE event SET comment=NULL WHERE objid=?", rid1); err != nil { t.Fatalf("UPDATE event SET comment=NULL: %v", err) } entries, err := Log(r, LogOpts{Start: rid1}) if err != nil { t.Fatalf("Log: %v", err) } if len(entries) != 1 { t.Fatalf("count = %d, want 1", len(entries)) } if entries[0].Comment != "" { t.Fatalf("entries[0].Comment = %q, want empty string for NULL event.comment", entries[0].Comment) } if entries[0].User != "testuser" { t.Fatalf("entries[0].User = %q, want %q", entries[0].User, "testuser") } } func TestCheckinDelta(t *testing.T) { r := setupTestRepo(t) rid1, _, _ := Checkin(r, CheckinOpts{ Files: []File{ {Name: "a.txt", Content: []byte("aaa")}, {Name: "b.txt", Content: []byte("bbb")}, }, Comment: "baseline", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) rid2, _, err := Checkin(r, CheckinOpts{ Files: []File{ {Name: "a.txt", Content: []byte("aaa-modified")}, {Name: "b.txt", Content: []byte("bbb")}, }, Comment: "delta", User: "testuser", Parent: rid1, Delta: true, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin delta: %v", err) } d, _ := GetManifest(r, rid2) if d.B == "" { t.Fatal("delta should have B-card") } if len(d.F) != 1 || d.F[0].Name != "a.txt" { t.Fatalf("delta F = %+v, want only a.txt", d.F) } files, _ := ListFiles(r, rid2) if len(files) != 2 { t.Fatalf("ListFiles = %d, want 2", len(files)) } } func TestCheckinDeltaFossilRebuild(t *testing.T) { testutil.RequireFossilBin(t) r := setupTestRepo(t) rid1, _, _ := Checkin(r, CheckinOpts{ Files: []File{ {Name: "a.txt", Content: []byte("aaa")}, {Name: "b.txt", Content: []byte("bbb")}, }, Comment: "baseline", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) _, _, err := Checkin(r, CheckinOpts{ Files: []File{ {Name: "a.txt", Content: []byte("aaa-v2")}, {Name: "b.txt", Content: []byte("bbb")}, {Name: "c.txt", Content: []byte("new file")}, }, Comment: "delta with add", User: "testuser", Parent: rid1, Delta: true, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("delta: %v", err) } r.Close() if err := testutil.FossilRebuild(r.Path()); err != nil { t.Fatalf("rebuild: %v", err) } } func TestCheckinWithCustomTags(t *testing.T) { r := setupTestRepo(t) // Initial checkin (gets default trunk tags) rid1, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "file.txt", Content: []byte("v1")}}, Comment: "initial", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("first checkin: %v", err) } // Second checkin with custom tags (branch creation) rid2, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "file.txt", Content: []byte("v2")}}, Comment: "branch commit", User: "testuser", Parent: rid1, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), Tags: []deck.TagCard{ {Type: deck.TagPropagating, Name: "branch", UUID: "*", Value: "feature-x"}, {Type: deck.TagSingleton, Name: "sym-feature-x", UUID: "*"}, }, }) if err != nil { t.Fatalf("second checkin: %v", err) } d, err := GetManifest(r, rid2) if err != nil { t.Fatalf("GetManifest: %v", err) } if len(d.T) != 2 { t.Fatalf("expected 2 tags, got %d: %+v", len(d.T), d.T) } // Verify the propagating branch tag found := false for _, tag := range d.T { if tag.Type == deck.TagPropagating && tag.Name == "branch" && tag.Value == "feature-x" { found = true } } if !found { t.Fatalf("missing propagating branch tag in %+v", d.T) } // Verify the singleton sym tag found = false for _, tag := range d.T { if tag.Type == deck.TagSingleton && tag.Name == "sym-feature-x" { found = true } } if !found { t.Fatalf("missing singleton sym-feature-x tag in %+v", d.T) } } func BenchmarkCheckin(b *testing.B) { path := filepath.Join(b.TempDir(), "bench.fossil") r, _ := repo.Create(path, "bench", simio.CryptoRand{}, "") defer r.Close() files := make([]File, 10) for i := range files { files[i] = File{Name: fmt.Sprintf("src/file%03d.go", i), Content: bytes.Repeat([]byte("x"), 1000)} } b.ResetTimer() var lastRid libfossil.FslID for i := 0; i < b.N; i++ { rid, _, _ := Checkin(r, CheckinOpts{ Files: files, Comment: "bench", User: "bench", Parent: lastRid, Time: time.Now().UTC(), }) lastRid = rid } } func BenchmarkListFiles(b *testing.B) { path := filepath.Join(b.TempDir(), "bench.fossil") r, _ := repo.Create(path, "bench", simio.CryptoRand{}, "") defer r.Close() files := make([]File, 50) for i := range files { files[i] = File{Name: fmt.Sprintf("src/file%03d.go", i), Content: []byte(fmt.Sprintf("c-%d", i))} } rid, _, _ := Checkin(r, CheckinOpts{ Files: files, Comment: "bench", User: "bench", Time: time.Now().UTC(), }) b.ResetTimer() for i := 0; i < b.N; i++ { ListFiles(r, rid) } } func TestCrosslinkInlineTCards(t *testing.T) { r := setupTestRepo(t) rid, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "hello.txt", Content: []byte("hello")}}, Comment: "initial commit", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin: %v", err) } // Clear event and tagxref to simulate post-clone state (blobs exist, but not crosslinked). r.DB().Exec("DELETE FROM event") r.DB().Exec("DELETE FROM plink") r.DB().Exec("DELETE FROM leaf") r.DB().Exec("DELETE FROM mlink") r.DB().Exec("DELETE FROM tagxref") // Run Crosslink. n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } t.Logf("crosslinked %d artifacts", n) // Verify inline T-cards were processed. // Initial checkin adds: *branch trunk, +sym-trunk var branchCount int r.DB().QueryRow( "SELECT count(*) FROM tagxref JOIN tag USING(tagid) WHERE tagname='branch' AND rid=?", rid, ).Scan(&branchCount) if branchCount != 1 { t.Errorf("branch tagxref count=%d, want 1", branchCount) } var symCount int r.DB().QueryRow( "SELECT count(*) FROM tagxref JOIN tag USING(tagid) WHERE tagname='sym-trunk' AND rid=?", rid, ).Scan(&symCount) if symCount != 1 { t.Errorf("sym-trunk tagxref count=%d, want 1", symCount) } } func TestCrosslinkControlArtifact(t *testing.T) { r := setupTestRepo(t) rid, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "hello.txt", Content: []byte("hello")}}, Comment: "initial commit", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin: %v", err) } // Create a control artifact via AddTag. _, err = tag.AddTag(r, tag.TagOpts{ TargetRID: rid, TagName: "testlabel", TagType: tag.TagSingleton, Value: "myvalue", User: "testuser", Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("AddTag: %v", err) } // Clear tagxref to simulate post-clone. r.DB().Exec("DELETE FROM tagxref") n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } t.Logf("crosslinked %d artifacts", n) // Verify control artifact's tag was re-applied. var count int r.DB().QueryRow( "SELECT count(*) FROM tagxref JOIN tag USING(tagid) WHERE tagname='testlabel' AND rid=?", rid, ).Scan(&count) if count != 1 { t.Errorf("testlabel tagxref count=%d, want 1", count) } } func TestCrosslinkControlEventRow(t *testing.T) { r := setupTestRepo(t) // Create a checkin to target rid, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "hello.txt", Content: []byte("hello")}}, Comment: "initial commit", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin: %v", err) } // Create a control artifact via AddTag controlRID, err := tag.AddTag(r, tag.TagOpts{ TargetRID: rid, TagName: "testlabel", TagType: tag.TagSingleton, Value: "myvalue", User: "testuser", Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("AddTag: %v", err) } // Clear tagxref and event to simulate post-clone r.DB().Exec("DELETE FROM tagxref") r.DB().Exec("DELETE FROM event WHERE objid=?", controlRID) // Run Crosslink n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } // Verify event row (type='g') exists for the control artifact var eventType, comment string err = r.DB().QueryRow( "SELECT type, comment FROM event WHERE objid=?", controlRID, ).Scan(&eventType, &comment) if err != nil { t.Fatalf("event query: %v", err) } if eventType != "g" { t.Errorf("event type=%q, want 'g'", eventType) } if comment == "" { t.Errorf("event comment is empty, expected descriptive text") } t.Logf("event comment: %q", comment) } func TestDiscoveryQueryIdempotent(t *testing.T) { r := setupTestRepo(t) // Create a checkin. rid, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "hello.txt", Content: []byte("hello")}}, Comment: "initial commit", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin: %v", err) } // Clear event and tagxref to simulate post-clone state. r.DB().Exec("DELETE FROM event") r.DB().Exec("DELETE FROM plink") r.DB().Exec("DELETE FROM leaf") r.DB().Exec("DELETE FROM mlink") r.DB().Exec("DELETE FROM tagxref") // First run should link artifacts. n1, err := Crosslink(r) if err != nil { t.Fatalf("first Crosslink: %v", err) } if n1 < 1 { t.Fatalf("first run linked %d, want >= 1", n1) } t.Logf("first run linked %d artifacts", n1) // Verify checkin was crosslinked. var eventCount int r.DB().QueryRow("SELECT count(*) FROM event WHERE objid=?", rid).Scan(&eventCount) if eventCount != 1 { t.Errorf("event count=%d, want 1", eventCount) } // Second run should link nothing (idempotent). n2, err := Crosslink(r) if err != nil { t.Fatalf("second Crosslink: %v", err) } if n2 != 0 { t.Errorf("second run linked %d, want 0 (idempotent)", n2) } t.Logf("second run linked %d artifacts (idempotent)", n2) } func TestCrosslinkCherrypick(t *testing.T) { r := setupTestRepo(t) // Create initial checkin rid1, uuid1, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "file.txt", Content: []byte("v1")}}, Comment: "first", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("first Checkin: %v", err) } // Create second checkin (parent of cherrypick target) rid2, uuid2, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "file.txt", Content: []byte("v2")}}, Comment: "second", User: "testuser", Parent: rid1, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("second Checkin: %v", err) } // Manually create a manifest with Q-card (cherrypick) // Since Checkin doesn't support Q-cards, we construct the deck manually d := &deck.Deck{ Type: deck.Checkin, C: "cherrypick commit", D: time.Date(2024, 1, 15, 12, 0, 0, 0, time.UTC), U: deck.User("testuser"), P: []string{uuid1}, // parent is rid1 Q: []deck.CherryPick{ {Target: uuid2, IsBackout: false}, // cherrypick rid2 }, F: []deck.FileCard{ {Name: "file.txt", UUID: uuid2}, // use file from rid2 }, } // Compute R-card (manifest hash) d.R = "0000000000000000000000000000000000000000" // simplified for test // Marshal and store the manifest blob manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } rid3, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("Store manifest: %v", err) } // Clear crosslink tables to simulate post-clone r.DB().Exec("DELETE FROM event WHERE objid=?", rid3) r.DB().Exec("DELETE FROM plink WHERE cid=?", rid3) r.DB().Exec("DELETE FROM mlink WHERE mid=?", rid3) r.DB().Exec("DELETE FROM cherrypick WHERE childid=?", rid3) // Run Crosslink n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } // Verify cherrypick table var count int r.DB().QueryRow( "SELECT count(*) FROM cherrypick WHERE parentid=? AND childid=? AND isExclude=0", rid2, rid3, ).Scan(&count) if count != 1 { t.Errorf("cherrypick count=%d, want 1", count) } } func TestCrosslinkWiki(t *testing.T) { r := setupTestRepo(t) // Manually create a wiki manifest wikiTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC) wikiContent := []byte("This is a test wiki page") d := &deck.Deck{ Type: deck.Wiki, L: "TestPage", U: deck.User("testuser"), W: wikiContent, D: wikiTime, } // Marshal and store the wiki manifest blob manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } rid, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("Store manifest: %v", err) } // Clear crosslink tables to simulate post-clone r.DB().Exec("DELETE FROM event WHERE objid=?", rid) r.DB().Exec("DELETE FROM tagxref WHERE rid=?", rid) // Run Crosslink n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } // Verify event row (type='w') var eventType, comment string err = r.DB().QueryRow( "SELECT type, comment FROM event WHERE objid=?", rid, ).Scan(&eventType, &comment) if err != nil { t.Fatalf("event query: %v", err) } if eventType != "w" { t.Errorf("event type=%q, want 'w'", eventType) } if comment != "+TestPage" { t.Errorf("event comment=%q, want '+TestPage'", comment) } // Verify wiki-TestPage tag in tagxref var tagCount int r.DB().QueryRow(` SELECT count(*) FROM tagxref JOIN tag USING(tagid) WHERE tagname='wiki-TestPage' AND rid=? `, rid).Scan(&tagCount) if tagCount != 1 { t.Errorf("wiki-TestPage tag count=%d, want 1", tagCount) } } func TestCrosslinkTicket(t *testing.T) { r := setupTestRepo(t) // Manually create a ticket manifest ticketTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC) ticketUUID := "0123456789abcdef0123456789abcdef01234567" d := &deck.Deck{ Type: deck.Ticket, K: ticketUUID, U: deck.User("testuser"), D: ticketTime, J: []deck.TicketField{ {Name: "title", Value: "Test ticket"}, {Name: "status", Value: "Open"}, }, } // Marshal and store the ticket manifest blob manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } rid, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("Store manifest: %v", err) } // Clear crosslink tables to simulate post-clone r.DB().Exec("DELETE FROM tagxref WHERE rid=?", rid) // Run Crosslink n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } // Verify tkt-* tag in tagxref var tagCount int r.DB().QueryRow(` SELECT count(*) FROM tagxref JOIN tag USING(tagid) WHERE tagname=? AND rid=? `, fmt.Sprintf("tkt-%s", ticketUUID), rid).Scan(&tagCount) if tagCount != 1 { t.Errorf("tkt-%s tag count=%d, want 1", ticketUUID, tagCount) } } func TestCrosslinkEvent(t *testing.T) { r := setupTestRepo(t) // Manually create an event (technote) manifest eventTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC) eventUUID := "fedcba9876543210fedcba9876543210fedcba98" eventBody := []byte("This is a technote body") d := &deck.Deck{ Type: deck.Event, E: &deck.EventCard{ Date: eventTime, UUID: eventUUID, }, U: deck.User("testuser"), D: eventTime, W: eventBody, C: "Test technote", } // Marshal and store the event manifest blob manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } rid, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("Store manifest: %v", err) } // Clear crosslink tables to simulate post-clone r.DB().Exec("DELETE FROM event WHERE objid=?", rid) r.DB().Exec("DELETE FROM tagxref WHERE rid=?", rid) // Run Crosslink n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } // Verify event row (type='e') var eventType, comment string err = r.DB().QueryRow( "SELECT type, comment FROM event WHERE objid=?", rid, ).Scan(&eventType, &comment) if err != nil { t.Fatalf("event query: %v", err) } if eventType != "e" { t.Errorf("event type=%q, want 'e'", eventType) } if comment != "Test technote" { t.Errorf("event comment=%q, want 'Test technote'", comment) } // Verify event-* tag in tagxref var tagCount int r.DB().QueryRow(` SELECT count(*) FROM tagxref JOIN tag USING(tagid) WHERE tagname=? AND rid=? `, fmt.Sprintf("event-%s", eventUUID), rid).Scan(&tagCount) if tagCount != 1 { t.Errorf("event-%s tag count=%d, want 1", eventUUID, tagCount) } } func TestCrosslinkAttachment(t *testing.T) { r := setupTestRepo(t) // Manually create an attachment manifest with wiki target attachTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC) d := &deck.Deck{ Type: deck.Attachment, A: &deck.AttachmentCard{ Filename: "test.txt", Target: "TestPage", Source: "abc123", }, U: deck.User("testuser"), D: attachTime, C: "Attach test file", } // Marshal and store the attachment manifest blob manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } rid, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("Store manifest: %v", err) } // Clear crosslink tables to simulate post-clone r.DB().Exec("DELETE FROM attachment WHERE attachid=?", rid) r.DB().Exec("DELETE FROM event WHERE objid=?", rid) // Run Crosslink n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } // Verify attachment table row var target, filename, src string var isLatestRaw any err = r.DB().QueryRow( "SELECT target, filename, src, isLatest FROM attachment WHERE attachid=?", rid, ).Scan(&target, &filename, &src, &isLatestRaw) if err != nil { t.Fatalf("attachment query: %v", err) } if target != "TestPage" { t.Errorf("target=%q, want 'TestPage'", target) } if filename != "test.txt" { t.Errorf("filename=%q, want 'test.txt'", filename) } if src != "abc123" { t.Errorf("src=%q, want 'abc123'", src) } isLatest, _ := db.ScanInt(isLatestRaw) if isLatest != 1 { t.Errorf("isLatest=%d, want 1", isLatest) } // Verify event row (type='w' for wiki attachment) var eventType, comment string err = r.DB().QueryRow( "SELECT type, comment FROM event WHERE objid=?", rid, ).Scan(&eventType, &comment) if err != nil { t.Fatalf("event query: %v", err) } if eventType != "w" { t.Errorf("event type=%q, want 'w'", eventType) } if comment != "Add attachment test.txt to wiki page TestPage" { t.Errorf("event comment=%q", comment) } } func TestCrosslinkCluster(t *testing.T) { r := setupTestRepo(t) // Store 2 blobs that will be members of the cluster blob1Data := []byte("test blob 1") rid1, uuid1, err := blob.Store(r.DB(), blob1Data) if err != nil { t.Fatalf("Store blob1: %v", err) } blob2Data := []byte("test blob 2") rid2, uuid2, err := blob.Store(r.DB(), blob2Data) if err != nil { t.Fatalf("Store blob2: %v", err) } // Verify both blobs are initially in unclustered table r.DB().Exec("INSERT OR IGNORE INTO unclustered(rid) VALUES(?)", rid1) r.DB().Exec("INSERT OR IGNORE INTO unclustered(rid) VALUES(?)", rid2) var count1, count2 int r.DB().QueryRow("SELECT count(*) FROM unclustered WHERE rid=?", rid1).Scan(&count1) r.DB().QueryRow("SELECT count(*) FROM unclustered WHERE rid=?", rid2).Scan(&count2) if count1 != 1 || count2 != 1 { t.Fatalf("blobs not in unclustered: rid1=%d rid2=%d", count1, count2) } // Create a cluster manifest with these two blobs as members clusterTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC) d := &deck.Deck{ Type: deck.Cluster, M: []string{uuid1, uuid2}, D: clusterTime, } // Marshal and store the cluster manifest blob manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } rid, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("Store manifest: %v", err) } // Clear crosslink tables to simulate post-clone r.DB().Exec("DELETE FROM tagxref WHERE rid=?", rid) // Run Crosslink n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } // Verify cluster tag var tagCount int r.DB().QueryRow(` SELECT count(*) FROM tagxref JOIN tag USING(tagid) WHERE tagname='cluster' AND rid=? `, rid).Scan(&tagCount) if tagCount != 1 { t.Errorf("cluster tag count=%d, want 1", tagCount) } // Verify both blobs were removed from unclustered table r.DB().QueryRow("SELECT count(*) FROM unclustered WHERE rid=?", rid1).Scan(&count1) r.DB().QueryRow("SELECT count(*) FROM unclustered WHERE rid=?", rid2).Scan(&count2) if count1 != 0 { t.Errorf("rid1 still in unclustered, count=%d", count1) } if count2 != 0 { t.Errorf("rid2 still in unclustered, count=%d", count2) } } func TestCrosslinkForum(t *testing.T) { r := setupTestRepo(t) // Create a forum post (thread starter) forumTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC) forumContent := []byte("This is a test forum post") d := &deck.Deck{ Type: deck.ForumPost, H: "Discussion about sync", U: deck.User("testuser"), W: forumContent, D: forumTime, } // Marshal and store the forum manifest blob manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } rid, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("Store manifest: %v", err) } // Clear crosslink tables to simulate post-clone r.DB().Exec("DELETE FROM event WHERE objid=?", rid) r.DB().Exec("DELETE FROM forumpost WHERE fpid=?", rid) // Run Crosslink n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } // Verify forumpost row with froot=self (thread starter) var froot int64 var fprev, firt interface{} var fmtime float64 err = r.DB().QueryRow( "SELECT froot, fprev, firt, fmtime FROM forumpost WHERE fpid=?", rid, ).Scan(&froot, &fprev, &firt, &fmtime) if err != nil { t.Fatalf("forumpost query: %v", err) } if froot != int64(rid) { t.Errorf("froot=%d, want %d (self)", froot, rid) } if fprev != nil { t.Errorf("fprev=%v, want nil (no previous edit)", fprev) } if firt != nil { t.Errorf("firt=%v, want nil (thread starter)", firt) } // Verify event row (type='f') var eventType, comment string err = r.DB().QueryRow( "SELECT type, comment FROM event WHERE objid=?", rid, ).Scan(&eventType, &comment) if err != nil { t.Fatalf("event query: %v", err) } if eventType != "f" { t.Errorf("event type=%q, want 'f'", eventType) } if comment != "Post: Discussion about sync" { t.Errorf("event comment=%q, want 'Post: Discussion about sync'", comment) } } // TestCrosslinkForumPostTableDropped reproduces #103: canonical `fossil // rebuild` drops on-demand tables it did not populate, including forumpost, // so a repository handed to us straight out of a rebuild may not have the // table at all. Crosslink must create it on demand the way canonical does, // rather than erroring with "no such table: forumpost". func TestCrosslinkForumPostTableDropped(t *testing.T) { r := setupTestRepo(t) // Simulate a repository straight out of a canonical rebuild that never // saw a forum post: the table does not exist at all. if _, err := r.DB().Exec("DROP TABLE forumpost"); err != nil { t.Fatalf("drop forumpost: %v", err) } forumTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC) d := &deck.Deck{ Type: deck.ForumPost, H: "Discussion about sync", U: deck.User("testuser"), W: []byte("This is a test forum post"), D: forumTime, } manifestBytes, err := d.Marshal() if err != nil { t.Fatalf("Marshal: %v", err) } rid, _, err := blob.Store(r.DB(), manifestBytes) if err != nil { t.Fatalf("Store manifest: %v", err) } n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 1 { t.Fatalf("expected at least 1 artifact crosslinked, got %d", n) } var froot int64 if err := r.DB().QueryRow("SELECT froot FROM forumpost WHERE fpid=?", rid).Scan(&froot); err != nil { t.Fatalf("forumpost query: %v", err) } if froot != int64(rid) { t.Errorf("froot=%d, want %d (self)", froot, rid) } } func TestCrosslinkTwoPass(t *testing.T) { r := setupTestRepo(t) // Store a wiki manifest blob (Type=Wiki, L="TwoPassPage", U="test", W=content, D=time). wikiTime := time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC) wikiContent := []byte("Two-pass wiki content") wikiDeck := &deck.Deck{ Type: deck.Wiki, L: "TwoPassPage", U: deck.User("testuser"), W: wikiContent, D: wikiTime, } wikiBytes, err := wikiDeck.Marshal() if err != nil { t.Fatalf("Marshal wiki: %v", err) } wikiRid, _, err := blob.Store(r.DB(), wikiBytes) if err != nil { t.Fatalf("Store wiki: %v", err) } // Create a checkin via Checkin(). checkinRid, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "file.txt", Content: []byte("checkin content")}}, Comment: "test checkin", User: "testuser", Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin: %v", err) } // Clear all event and tagxref rows to simulate post-clone state. r.DB().Exec("DELETE FROM event") r.DB().Exec("DELETE FROM tagxref") r.DB().Exec("DELETE FROM plink") r.DB().Exec("DELETE FROM leaf") r.DB().Exec("DELETE FROM mlink") // Call Crosslink once. n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n < 2 { t.Fatalf("expected n >= 2 artifacts crosslinked, got %d", n) } t.Logf("crosslinked %d artifacts", n) // Verify: wiki event type='w' exists. var wikiEventType string err = r.DB().QueryRow("SELECT type FROM event WHERE objid=?", wikiRid).Scan(&wikiEventType) if err != nil { t.Errorf("wiki event query: %v", err) } else if wikiEventType != "w" { t.Errorf("wiki event type=%q, want 'w'", wikiEventType) } // Verify: checkin event type='ci' exists. var checkinEventType string err = r.DB().QueryRow("SELECT type FROM event WHERE objid=?", checkinRid).Scan(&checkinEventType) if err != nil { t.Errorf("checkin event query: %v", err) } else if checkinEventType != "ci" { t.Errorf("checkin event type=%q, want 'ci'", checkinEventType) } } // forceDeltify rewrites rid as a delta against srcRid, the same way // content.Deltify would, without content.Deltify's size/ratio gate -- // exercising that gate isn't the point of the tests that use this, only // forcing candidates into a genuine delta relationship so Crosslink's // delta-chain-order sweep has something to reorder around. Mirrors // deltifyWrite (internal/content/deltify.go) exactly: compress the delta // bytes over the blob row, link it in delta, and verify the round trip // through content.Expand before trusting it. func forceDeltify(t *testing.T, r *repo.Repo, rid, srcRid libfossil.FslID) { t.Helper() data, err := content.Expand(r.DB(), rid) if err != nil { t.Fatalf("forceDeltify: expand rid=%d: %v", rid, err) } src, err := content.Expand(r.DB(), srcRid) if err != nil { t.Fatalf("forceDeltify: expand srcid=%d: %v", srcRid, err) } deltaBytes := delta.Create(src, data) compressed, err := blob.Compress(deltaBytes) if err != nil { t.Fatalf("forceDeltify: compress: %v", err) } if err := r.WithTx(func(tx *db.Tx) error { if _, err := tx.Exec("UPDATE blob SET content=? WHERE rid=?", compressed, rid); err != nil { return err } _, err := tx.Exec("REPLACE INTO delta(rid, srcid) VALUES(?, ?)", rid, srcRid) return err }); err != nil { t.Fatalf("forceDeltify: %v", err) } if _, err := content.Expand(r.DB(), rid); err != nil { t.Fatalf("forceDeltify: round-trip verify rid=%d: %v", rid, err) } } // TestCrosslinkReversedDeltaOrderStillPropagatesAndLeafs is the #102 // regression: manifest.Checkin deltifies each checkin's parent manifest // against the new one it just wrote (content.Deltify(parent, this)), so in // a repository old enough for that to take, the chain root is the newest // commit and the deepest ancestor is the oldest -- forced here directly // with forceDeltify rather than relying on real commits being large and // similar enough for content.Deltify's own size/ratio gate to take the // delta, which is a orthogonal concern this test doesn't need to exercise. // Once the derived tables are cleared to simulate a fresh clone, Crosslink's // delta-chain-order sweep visits v3 first and v1 last -- the exact reverse // of commit order, and the opposite of what the old per-checkin // PropagateAll(parent) and incremental leaf bookkeeping assumed. tagxref and // leaf must still come out correct: v1's propagating branch tag on all // three, and only v3 marked a leaf. func TestCrosslinkReversedDeltaOrderStillPropagatesAndLeafs(t *testing.T) { r := setupTestRepo(t) v1, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v1 content")}}, Comment: "v1", User: "testuser", Time: time.Date(2024, 1, 15, 10, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin v1: %v", err) } v2, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v2 content")}}, Comment: "v2", User: "testuser", Parent: v1, Time: time.Date(2024, 1, 15, 11, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin v2: %v", err) } v3, _, err := Checkin(r, CheckinOpts{ Files: []File{{Name: "a.txt", Content: []byte("v3 content")}}, Comment: "v3", User: "testuser", Parent: v2, Time: time.Date(2024, 1, 15, 12, 0, 0, 0, time.UTC), }) if err != nil { t.Fatalf("Checkin v3: %v", err) } forceDeltify(t, r, v1, v2) forceDeltify(t, r, v2, v3) // Reset to the state a fresh clone is in right before crosslinking. for _, tbl := range crosslinkDerivedTables { if _, err := r.DB().Exec("DELETE FROM " + tbl); err != nil { t.Fatalf("clear %s: %v", tbl, err) } } n, err := Crosslink(r) if err != nil { t.Fatalf("Crosslink: %v", err) } if n != 3 { t.Fatalf("linked=%d, want 3", n) } for _, rid := range []libfossil.FslID{v1, v2, v3} { var value string err := r.DB().QueryRow(` SELECT value FROM tagxref JOIN tag USING(tagid) WHERE tagname='branch' AND rid=? `, rid).Scan(&value) if err != nil { t.Errorf("rid=%d: branch tag missing: %v", rid, err) } else if value != "trunk" { t.Errorf("rid=%d: branch tag=%q, want %q", rid, value, "trunk") } } var leafRids []libfossil.FslID rows, err := r.DB().Query("SELECT rid FROM leaf ORDER BY rid") if err != nil { t.Fatalf("leaf query: %v", err) } for rows.Next() { var rid libfossil.FslID if err := rows.Scan(&rid); err != nil { t.Fatalf("leaf scan: %v", err) } leafRids = append(leafRids, rid) } rows.Close() if len(leafRids) != 1 || leafRids[0] != v3 { t.Errorf("leaf=%v, want [%d] (v3 only)", leafRids, v3) } }