Artifact [94d2e17683]

Artifact 94d2e17683bd04c5f10a3672a1577f278ebd78ad496a7a06642e2c2254f96c07:

• File internal/blob/inflater.go — part of check-in [a2bcca2535] at 2026-07-24 14:34:17 on branch trunk — content: cut sub-64KiB delta-chain expansion allocation ~18x (#145) (#150)

  delta.Apply's one-time capacity upgrade (#127) only engages above the 64 KiB threshold, so it is a no-op for the typical source file under 64 KiB. But below the threshold Apply already sizes its output buffer exactly, so there was never a 5x overhead there to remove -- extending the threshold buys nothing. The real per-link cost of a chain walk was elsewhere, and a profile of a plain Expand of a deep sub-64 KiB chain shows it: a fresh ~32 KiB zlib history window allocated per delta load (~27% of bytes), one full-size Apply output buffer per link (~16%), and a fresh inflate buffer per link. All three are O(depth) allocations for a single cold Expand, which is O(depth), making a full-history walk O(n^2).

  Attack the per-link constant inside the one choke point every expansion goes through, content.expandChain, leaving the on-disk chain shape untouched:

  - delta.ApplyInto(dst, source, delta) reconstructs into a caller-supplied buffer, reusing its capacity. Apply is now ApplyInto(nil, ...), unchanged. A reused buffer's capacity is proven prior work, not a wire-header claim, so it is a safe starting size; the anti-DoS growth path (growOutputCap, keyed off len(delta) not the targetLen header) is preserved for buffers a target outgrows. - blob.Inflater loads a run of blobs through one zlib reader, reset per blob instead of rebuilt, paying the ~32 KiB window setup once per chain. Each Load still returns a freshly owned buffer, so no returned-slice-aliasing contract leaks to callers. - expandChain's plain-Expand replay (put == nil) ping-pongs two output buffers -- each link's target reuses the buffer from two links back, whose content was already consumed as the previous link's source, so dst and source never alias. The caching path (put != nil) is unchanged: put takes ownership of every materialized buffer, so those cannot be reused underneath it, but it still shares the reused zlib reader for its delta loads.

  Measured on Apple M4 Pro, BenchmarkExpandOldestVersion, median of 3, 40x:

  sub-64 KiB (~26 KB file), Expand of the oldest revision: depth 11: 883 KB -> 232 KB/op (3.8x), 1.12 ms -> 0.90 ms depth 49: 3.57 MB -> 341 KB/op (10.5x), 4.55 ms -> 2.03 ms depth 199: 14.2 MB -> 772 KB/op (18.4x), 14.84 ms -> 7.25 ms

  >64 KiB (~206 KB file), no regression -- ping-pong + reader reuse help it too: depth 11: 4.11 MB -> 1.27 MB/op (3.2x), 3.40 ms -> 2.74 ms depth 49: 16.5 MB -> 1.81 MB/op (9.1x), 13.02 ms -> 7.69 ms depth 199: 65.2 MB -> 3.92 MB/op (16.6x), 46.3 ms -> 26.6 ms

  Sub-64 KiB allocation is now near-flat in depth instead of linear in depth*targetLen. expand_reuse_test.go guards the reuse; delta and blob gain allocation-reduction unit tests. (user: [email protected], size: 2647) [annotate] [blame] [check-ins using]

package blob

import (
	"bytes"
	"compress/zlib"
	"fmt"
	"io"

	"github.com/danmestas/go-libfossil/db"
	libfossil "github.com/danmestas/go-libfossil/internal/fsltype"
	"github.com/danmestas/go-libfossil/simio"
)

// Inflater loads a run of blobs through a single reusable zlib reader. The
// zero value is ready to use. It is not safe for concurrent use.
//
// It exists for delta-chain replay, where one Expand inflates every delta on a
// chain in sequence. Building a fresh zlib reader per link re-allocates a
// ~32 KiB flate history window and its machinery each time, and that per-link
// setup was the single largest allocation source in sub-64 KiB chain
// expansion. Resetting one reader instead pays the setup once for the whole
// chain. Each Load still returns a freshly allocated buffer the caller owns —
// only the reader is shared, so results never alias one another.
type Inflater struct {
	zr io.ReadCloser
}

// Load returns rid's fully-expanded content, decompressing through the reused
// reader. It behaves exactly like [Load] but amortizes the reader setup across
// calls.
func (inf *Inflater) Load(q db.Querier, rid libfossil.FslID) ([]byte, error) {
	if inf == nil {
		panic("blob.Inflater.Load: inf must not be nil")
	}
	return loadWith(q, rid, inf)
}

// decompress mirrors [Decompress] but reuses inf.zr across calls via zlib's
// Resetter, building the reader only on the first call. Reading to EOF
// verifies the Adler-32 checksum, so no Close is needed between resets.
func (inf *Inflater) decompress(data []byte) (result []byte, err error) {
	if data == nil {
		panic("blob.Inflater.decompress: data must not be nil")
	}
	defer func() {
		if err == nil && result == nil {
			panic("blob.Inflater.decompress: postcondition violated: result is nil with no error")
		}
	}()

	if len(data) < 5 {
		return nil, fmt.Errorf("zlib decompress: data too short (%d bytes)", len(data))
	}
	// Skip the 4-byte size prefix, matching Decompress.
	zlibData := data[4:]

	if inf.zr == nil {
		r, err := zlib.NewReader(bytes.NewReader(zlibData))
		if err != nil {
			return nil, fmt.Errorf("zlib decompress: %w", err)
		}
		inf.zr = r
	} else {
		if err := inf.zr.(zlib.Resetter).Reset(bytes.NewReader(zlibData), nil); err != nil {
			return nil, fmt.Errorf("zlib reset: %w", err)
		}
	}

	out, err := io.ReadAll(inf.zr)
	if err != nil {
		return nil, fmt.Errorf("zlib read: %w", err)
	}
	// BUGGIFY: truncate decompressed output to exercise partial-read handling,
	// mirroring Decompress so the reused-reader path is fuzzed the same way.
	if simio.Buggify(0.02) && len(out) > 1 {
		out = out[:len(out)/2]
	}
	return out, nil
}