Artifact [4a1e5c1db2]

Artifact 4a1e5c1db2d5fbb257743210c9f746e72d4ff59eda47609c057bb5cfcf4df918:

• File internal/blob/inflater_test.go — part of check-in [a2bcca2535] at 2026-07-24 14:34:17 on branch trunk — content: cut sub-64KiB delta-chain expansion allocation ~18x (#145) (#150)

  delta.Apply's one-time capacity upgrade (#127) only engages above the 64 KiB threshold, so it is a no-op for the typical source file under 64 KiB. But below the threshold Apply already sizes its output buffer exactly, so there was never a 5x overhead there to remove -- extending the threshold buys nothing. The real per-link cost of a chain walk was elsewhere, and a profile of a plain Expand of a deep sub-64 KiB chain shows it: a fresh ~32 KiB zlib history window allocated per delta load (~27% of bytes), one full-size Apply output buffer per link (~16%), and a fresh inflate buffer per link. All three are O(depth) allocations for a single cold Expand, which is O(depth), making a full-history walk O(n^2).

  Attack the per-link constant inside the one choke point every expansion goes through, content.expandChain, leaving the on-disk chain shape untouched:

  - delta.ApplyInto(dst, source, delta) reconstructs into a caller-supplied buffer, reusing its capacity. Apply is now ApplyInto(nil, ...), unchanged. A reused buffer's capacity is proven prior work, not a wire-header claim, so it is a safe starting size; the anti-DoS growth path (growOutputCap, keyed off len(delta) not the targetLen header) is preserved for buffers a target outgrows. - blob.Inflater loads a run of blobs through one zlib reader, reset per blob instead of rebuilt, paying the ~32 KiB window setup once per chain. Each Load still returns a freshly owned buffer, so no returned-slice-aliasing contract leaks to callers. - expandChain's plain-Expand replay (put == nil) ping-pongs two output buffers -- each link's target reuses the buffer from two links back, whose content was already consumed as the previous link's source, so dst and source never alias. The caching path (put != nil) is unchanged: put takes ownership of every materialized buffer, so those cannot be reused underneath it, but it still shares the reused zlib reader for its delta loads.

  Measured on Apple M4 Pro, BenchmarkExpandOldestVersion, median of 3, 40x:

  sub-64 KiB (~26 KB file), Expand of the oldest revision: depth 11: 883 KB -> 232 KB/op (3.8x), 1.12 ms -> 0.90 ms depth 49: 3.57 MB -> 341 KB/op (10.5x), 4.55 ms -> 2.03 ms depth 199: 14.2 MB -> 772 KB/op (18.4x), 14.84 ms -> 7.25 ms

  >64 KiB (~206 KB file), no regression -- ping-pong + reader reuse help it too: depth 11: 4.11 MB -> 1.27 MB/op (3.2x), 3.40 ms -> 2.74 ms depth 49: 16.5 MB -> 1.81 MB/op (9.1x), 13.02 ms -> 7.69 ms depth 199: 65.2 MB -> 3.92 MB/op (16.6x), 46.3 ms -> 26.6 ms

  Sub-64 KiB allocation is now near-flat in depth instead of linear in depth*targetLen. expand_reuse_test.go guards the reuse; delta and blob gain allocation-reduction unit tests. (user: [email protected], size: 2538) [annotate] [blame] [check-ins using]

package blob

import (
	"bytes"
	"testing"

	libfossil "github.com/danmestas/go-libfossil/internal/fsltype"
)

// TestInflater_LoadMatchesLoad pins the contract: an Inflater must return the
// same bytes as the plain Load for the same rid, on the first call and on
// every reuse of its reader. Goal: prove the reader-reuse path decompresses
// identically to the fresh-reader path, including the second call that
// actually exercises the reset.
func TestInflater_LoadMatchesLoad(t *testing.T) {
	d := setupTestDB(t)
	// Compressible payload large enough that Store keeps it zlib-compressed,
	// so Inflater.Load takes the decompression path under test.
	content := bytes.Repeat([]byte("delta chain payload line\n"), 500)
	rid, _, err := Store(d, content)
	if err != nil {
		t.Fatalf("Store: %v", err)
	}

	want, err := Load(d, rid)
	if err != nil {
		t.Fatalf("Load: %v", err)
	}

	var inf Inflater
	for call := 0; call < 3; call++ {
		got, err := inf.Load(d, rid)
		if err != nil {
			t.Fatalf("Inflater.Load call %d: %v", call, err)
		}
		if !bytes.Equal(got, want) {
			t.Fatalf("Inflater.Load call %d differs from Load", call)
		}
	}
}

// TestInflater_ReusesReaderAcrossLoads is the reason Inflater exists: loading a
// run of compressed blobs through one Inflater must allocate less than loading
// them through the plain Load, because the expensive zlib reader (a ~32 KiB
// history window plus its flate machinery) is built once and reset per blob
// rather than rebuilt each time. Goal: prove the reuse measurably lowers
// allocations over a sequence, measured with AllocsPerRun so it does not
// depend on the exact per-call count.
func TestInflater_ReusesReaderAcrossLoads(t *testing.T) {
	d := setupTestDB(t)

	const blobs = 8
	rids := make([]libfossil.FslID, blobs)
	for i := range rids {
		content := bytes.Repeat([]byte{byte('a' + i)}, 4096)
		rid, _, err := Store(d, content)
		if err != nil {
			t.Fatalf("Store %d: %v", i, err)
		}
		rids[i] = rid
	}

	plainAllocs := testing.AllocsPerRun(50, func() {
		for _, rid := range rids {
			if _, err := Load(d, rid); err != nil {
				t.Fatalf("Load: %v", err)
			}
		}
	})

	var inf Inflater
	reuseAllocs := testing.AllocsPerRun(50, func() {
		for _, rid := range rids {
			if _, err := inf.Load(d, rid); err != nil {
				t.Fatalf("Inflater.Load: %v", err)
			}
		}
	})

	if reuseAllocs >= plainAllocs {
		t.Fatalf("Inflater.Load allocated %.0f over %d blobs, want fewer than "+
			"plain Load's %.0f -- the reader was not reused",
			reuseAllocs, blobs, plainAllocs)
	}
}