Artifact 49c3879a4781a01b530ffbd84c31ab43d7be37df16f3b6df146b33b2a9d21142:
- File
internal/annotate/annotate_bench_test.go
— part of check-in
[0fd2a507b8]
at
2026-07-24 15:27:03
on branch trunk
— perf: route history-walking expansions through content.Cache (#151) (#153)
The buffer-reuse fix in #145/#150 cut the cost of expanding one delta chain once. It did nothing for callers that expand many versions of the same artifact in a single operation: each bare content.Expand still replayed the whole O(depth) chain, so a walk over N overlapping revisions cost O(N^2) blob reads even though content.Cache already existed for exactly this pattern.
Audited every non-test content.Expand caller. Two access patterns genuinely repeat over overlapping delta chains and are now routed through a per-operation content.Cache:
- annotate walks every revision of one file along the parent chain. content_deltify stores each older file blob as a delta against its successor, so the per-revision expansions are all walks of the same few chains. One cache threaded through the walk (file blob + manifest listing via new manifest.ListFilesCached/GetManifestCached, both nil-passthrough so existing callers are unchanged) turns it linear. Measured on a deltified 150-revision file: 1.60s -> 0.65s (2.5x), 764k -> 308k allocs; the win grows with history depth.
- verify.Verify and verify.Rebuild sweep every blob more than once (Rebuild: checkBlobs + rebuildManifests + rebuildTags x2). A single cache shared across the pass walks each chain once instead of once per sweep and amortizes overlap within each sweep. Measured on a 150-revision two-file repo: Rebuild 3.34s -> 0.068s (48x), 4.59M -> 98k allocs.
Callers that only ever touch the tip or make a single pass over distinct blobs (branch create, merge-config load, R-card / delta manifest build, MergeParentFiles, crosslinkSingle, shun undelta, pairwise Diff) are left on bare Expand, each with a comment stating why a cache cannot amortize their access.
No behavior change: expansion output is byte-identical, verified across the modernc and ncruces drivers with REQUIRE_FOSSIL_BIN=1 cross-checks and the DST suite. (user: [email protected], size: 3485) [annotate] [blame] [check-ins using]
package annotate import ( "fmt" "path/filepath" "testing" "time" libfossil "github.com/danmestas/go-libfossil/internal/fsltype" "github.com/danmestas/go-libfossil/internal/manifest" "github.com/danmestas/go-libfossil/internal/repo" _ "github.com/danmestas/go-libfossil/internal/testdriver" "github.com/danmestas/go-libfossil/simio" ) // BenchmarkAnnotateDeepHistory measures Annotate over a file with a long, // deltified revision history -- the exact shape issue #151 targets. // // Methodology: commit `revisions` versions of a ~26 KB, 400-line file, each // changing a strided subset of lines off the previous. manifest.Checkin // deltifies as it goes, storing every older file revision as a delta against // the one newer and leaving the newest whole (verified: the file blob at each // revision carries a delta.srcid to its successor). Annotate(tip) then walks // the whole parent chain, loading the file at every revision. Without a shared // cache each load replays that revision's delta chain from the whole tip: // O(revisions^2) delta applications. The per-op numbers (ns/op, B/op, // allocs/op) are what routing the walk through content.Cache is meant to move. func BenchmarkAnnotateDeepHistory(b *testing.B) { for _, revisions := range []int{20, 60, 150} { b.Run(fmt.Sprintf("revisions=%d", revisions), func(b *testing.B) { r, tip := buildFileHistory(b, revisions) b.ReportAllocs() b.ResetTimer() for i := 0; i < b.N; i++ { lines, err := Annotate(r, Options{FilePath: "file.txt", StartRID: tip}) if err != nil { b.Fatalf("Annotate: %v", err) } if len(lines) == 0 { b.Fatal("Annotate returned no lines") } } }) } } // buildFileHistory commits `revisions` versions of one file, each a small edit // off the last, and returns the repo and the tip checkin rid. Checkin deltifies // the file blobs into a backward chain (newest whole, older revisions deltas // against it) on its own, matching a real repository's on-disk shape. func buildFileHistory(b *testing.B, revisions int) (*repo.Repo, libfossil.FslID) { b.Helper() if revisions < 2 { panic("buildFileHistory: revisions must be >= 2") } path := filepath.Join(b.TempDir(), "hist.fossil") r, err := repo.Create(path, "bench", simio.CryptoRand{}, "") if err != nil { b.Fatalf("repo.Create: %v", err) } b.Cleanup(func() { r.Close() }) base := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC) var parent, tip libfossil.FslID for k := 0; k < revisions; k++ { rid, _, err := manifest.Checkin(r, manifest.CheckinOpts{ Files: []manifest.File{{Name: "file.txt", Content: benchRevision(k)}}, Comment: fmt.Sprintf("rev %d", k), User: "bench", Parent: parent, Time: base.Add(time.Duration(k) * time.Hour), }) if err != nil { b.Fatalf("checkin rev %d: %v", k, err) } parent = rid tip = rid } return r, tip } // benchRevision returns revision k's content: a deterministic ~400-line file // whose lines drift with k so consecutive revisions differ by a handful of // lines -- the shape content_deltify sees, and enough change per step that the // deltifier keeps each revision as a delta rather than a whole copy. func benchRevision(k int) []byte { const lines = 400 buf := make([]byte, 0, lines*66) for i := 0; i < lines; i++ { token := i if i%11 == 0 { token = i + k } buf = append(buf, fmt.Sprintf( "line %04d token %08d lorem ipsum dolor sit amet consectetur\n", i, token)...) } return buf }