Artifact [98f7477f94]

Artifact 98f7477f9410d4775564a0829160bcfb3ff27111a7f9ea28f28a1828f6c8156f:

• File internal/content/expand_reuse_test.go — part of check-in [a2bcca2535] at 2026-07-24 14:34:17 on branch trunk — content: cut sub-64KiB delta-chain expansion allocation ~18x (#145) (#150)

  delta.Apply's one-time capacity upgrade (#127) only engages above the 64 KiB threshold, so it is a no-op for the typical source file under 64 KiB. But below the threshold Apply already sizes its output buffer exactly, so there was never a 5x overhead there to remove -- extending the threshold buys nothing. The real per-link cost of a chain walk was elsewhere, and a profile of a plain Expand of a deep sub-64 KiB chain shows it: a fresh ~32 KiB zlib history window allocated per delta load (~27% of bytes), one full-size Apply output buffer per link (~16%), and a fresh inflate buffer per link. All three are O(depth) allocations for a single cold Expand, which is O(depth), making a full-history walk O(n^2).

  Attack the per-link constant inside the one choke point every expansion goes through, content.expandChain, leaving the on-disk chain shape untouched:

  - delta.ApplyInto(dst, source, delta) reconstructs into a caller-supplied buffer, reusing its capacity. Apply is now ApplyInto(nil, ...), unchanged. A reused buffer's capacity is proven prior work, not a wire-header claim, so it is a safe starting size; the anti-DoS growth path (growOutputCap, keyed off len(delta) not the targetLen header) is preserved for buffers a target outgrows. - blob.Inflater loads a run of blobs through one zlib reader, reset per blob instead of rebuilt, paying the ~32 KiB window setup once per chain. Each Load still returns a freshly owned buffer, so no returned-slice-aliasing contract leaks to callers. - expandChain's plain-Expand replay (put == nil) ping-pongs two output buffers -- each link's target reuses the buffer from two links back, whose content was already consumed as the previous link's source, so dst and source never alias. The caching path (put != nil) is unchanged: put takes ownership of every materialized buffer, so those cannot be reused underneath it, but it still shares the reused zlib reader for its delta loads.

  Measured on Apple M4 Pro, BenchmarkExpandOldestVersion, median of 3, 40x:

  sub-64 KiB (~26 KB file), Expand of the oldest revision: depth 11: 883 KB -> 232 KB/op (3.8x), 1.12 ms -> 0.90 ms depth 49: 3.57 MB -> 341 KB/op (10.5x), 4.55 ms -> 2.03 ms depth 199: 14.2 MB -> 772 KB/op (18.4x), 14.84 ms -> 7.25 ms

  >64 KiB (~206 KB file), no regression -- ping-pong + reader reuse help it too: depth 11: 4.11 MB -> 1.27 MB/op (3.2x), 3.40 ms -> 2.74 ms depth 49: 16.5 MB -> 1.81 MB/op (9.1x), 13.02 ms -> 7.69 ms depth 199: 65.2 MB -> 3.92 MB/op (16.6x), 46.3 ms -> 26.6 ms

  Sub-64 KiB allocation is now near-flat in depth instead of linear in depth*targetLen. expand_reuse_test.go guards the reuse; delta and blob gain allocation-reduction unit tests. (user: [email protected], size: 1984) [annotate] [blame] [check-ins using]

package content

import (
	"bytes"
	"runtime"
	"testing"
)

// TestExpand_DeepChainReusesBuffers is the regression guard for issue #145's
// sub-64 KiB fix. Goal: prove a plain Expand of a deep backward chain no longer
// allocates one full-size output buffer per link.
//
// Methodology: build a ~26 KB file (below delta.Apply's 64 KiB cap) with a
// 60-revision backward delta chain, expand the oldest rid once to warm and to
// capture the correct bytes, then measure TotalAlloc across a second Expand.
// Before the fix each of the ~59 links allocated its own full-size target
// buffer (plus a fresh zlib reader), landing near 2.7x depth*targetLen. The
// ping-pong replay holds a constant two output buffers and reuses one zlib
// reader, so total allocation must stay under a single full buffer per link.
func TestExpand_DeepChainReusesBuffers(t *testing.T) {
	d := setupBenchDB(t)
	const revisions = 60
	oldest, depth := buildBackwardChain(t, d, revisions, smallFileLines)

	want, err := Expand(d, oldest)
	if err != nil {
		t.Fatalf("Expand (warm): %v", err)
	}
	targetLen := len(want)

	runtime.GC()
	var before, after runtime.MemStats
	runtime.ReadMemStats(&before)

	got, err := Expand(d, oldest)

	runtime.ReadMemStats(&after)

	if err != nil {
		t.Fatalf("Expand: %v", err)
	}
	if !bytes.Equal(got, want) {
		t.Fatalf("Expand of a depth-%d chain returned wrong content", depth)
	}

	grown := after.TotalAlloc - before.TotalAlloc
	// One full output buffer per link is the pre-fix floor for Apply outputs
	// alone; counting the per-link zlib window the old path landed well above
	// it. Staying under this bound proves the outputs and reader are reused.
	maxBytes := uint64(depth) * uint64(targetLen)
	if grown > maxBytes {
		t.Fatalf("Expand of a depth-%d chain allocated %d bytes (~%.1fx a %d-byte target); "+
			"want under %d (one full buffer per link) -- buffers are not being reused",
			depth, grown, float64(grown)/float64(targetLen), targetLen, maxBytes)
	}
}