Artifact 6f94eccb24e0affed867e88dc60d188618f21bf4ff60f42af5a04d1318d17438:
- File
internal/blob/compress_test.go
— part of check-in
[cc42ad8829]
at
2026-07-23 09:52:23
on branch trunk
— sync: store received wire frames verbatim instead of re-encoding (#132)
* fix: store received wire frames verbatim instead of re-encoding
Clone reproduced content correctly but not bytes: the receive path decompressed every cfile frame and re-compressed it with blob.Compress's unparameterized zlib writer, so a clone's blob.content never matched its source's — about 0.5% larger on the wapp corpus, purely from re-encoding frames that were already correctly encoded on the wire.
xfer's cfile decoder now hands back StoredBlob alongside the decompressed Content: the same bytes recast into Fossil's on-disk blob format ([4-byte BE size][zlib data]) without a second zlib pass — verbatim when the wire already sent that format (Fossil's send_compressed_file case), or the received zlib stream with a length prefix bolted on when it didn't (our own encoder's plain-zlib case). storeReceivedFile/storeDeltaContent/ storeResolvedContent and blob.StoreDeltaRaw thread that through via a new blob.EncodeForStorage helper, which writes the verbatim bytes when given and only falls back to Compress when there's nothing to preserve — covering both the clone/pull and server-push receive paths.
Locally authored content (never received over the wire) always takes the Compress fallback, so #71's deltification policy is untouched. The clone BUGGIFY corruption test drops storedBlob alongside the corrupted content it's exercising, so verbatim storage can't mask the corruption it exists to catch.
Adds a corpus-level regression test cloning a real multi-revision fossil repo and asserting sum(length(content)) matches between source and clone.
* test: use RequireFossilBin instead of removed HasFossil helper
testutil.HasFossil was removed upstream in favor of RequireFossilBin; this test's skip guard needed the same update to compile against current main. (user: [email protected], size: 2567) [annotate] [blame] [check-ins using]
package blob import ( "bytes" "testing" ) // TestEncodeForStorageUsesVerbatimWhenGiven is the storage-layer regression // test for issue #112: a caller offering already wire-encoded bytes must // get exactly those bytes back, not a fresh Compress() of data. This is // what lets a receive path skip re-encoding entirely rather than compress // then discard. func TestEncodeForStorageUsesVerbatimWhenGiven(t *testing.T) { data := []byte("content whose caller already has an encoded wire form") verbatim := []byte("not a real zlib stream, but EncodeForStorage must not care") got, err := EncodeForStorage(data, verbatim) if err != nil { t.Fatalf("EncodeForStorage: %v", err) } if !bytes.Equal(got, verbatim) { t.Fatalf("EncodeForStorage did not return verbatim bytes unchanged:\n got %q\n want %q", got, verbatim) } } // TestEncodeForStorageCompressesWhenVerbatimNil covers the fallback used by // locally authored content, which never has wire-encoded bytes to offer. func TestEncodeForStorageCompressesWhenVerbatimNil(t *testing.T) { data := []byte("locally authored content, never received over the wire") got, err := EncodeForStorage(data, nil) if err != nil { t.Fatalf("EncodeForStorage: %v", err) } want, err := Compress(data) if err != nil { t.Fatalf("Compress: %v", err) } if !bytes.Equal(got, want) { t.Fatalf("EncodeForStorage(nil verbatim) != Compress(data)") } decoded, err := Decompress(got) if err != nil { t.Fatalf("Decompress: %v", err) } if !bytes.Equal(decoded, data) { t.Fatalf("round trip failed: got %q, want %q", decoded, data) } } // TestEncodeForStorageRejectsNilData asserts the documented precondition: // data is a programmer-controlled argument (unlike verbatim, which is wire // data), so a nil data slice is a caller bug, not a wire-error to report. func TestEncodeForStorageRejectsNilData(t *testing.T) { defer func() { if recover() == nil { t.Fatal("EncodeForStorage(nil data, ...) did not panic") } }() EncodeForStorage(nil, []byte("irrelevant")) } // TestEncodeForStorageRejectsEmptyVerbatim guards against a caller passing // a non-nil-but-empty slice, which would otherwise silently mean "use // verbatim" while storing zero bytes -- an empty blob.content is never a // valid encoding, so this must fail loudly rather than corrupt storage. func TestEncodeForStorageRejectsEmptyVerbatim(t *testing.T) { defer func() { if recover() == nil { t.Fatal("EncodeForStorage(data, empty non-nil verbatim) did not panic") } }() EncodeForStorage([]byte("data"), []byte{}) }