Artifact ce1fc8919cbe3434cbf2d08d60bd060c2a6bbb9ea87a1b62fb6041cc82d14a80:
- File
internal/blob/doc.go
— part of check-in
[1fd12b3ed3]
at
2026-10-28 21:53:09
on branch feat/chunked-blobs
— Store blobs over a hosted value limit across blobpart rows
A hosted SQLite that caps value size (a Durable Object's 2 MB) cannot hold a large file's stored blob in one blob.content value. With the new db.DriverConfig.MaxValueBytes set, a stored value over the limit is written in 1,000,000-byte parts to blobpart(rid, seq, content), created on first use, leaving blob.content NULL and blob.size the expanded length, so phantoms (size < 0) stay unambiguous. Chunking sits below compression: hashes, deltas and the wire format are unchanged.
internal/blob owns the seam (ReadStored, WriteStored, InsertStored, StoredHead, DropStored, SweepParts and two scan expressions), and every direct reader or writer of blob.content now goes through it: delta rewrite and undelta, receive, clone send and walk, compaction, sparse demotion, seed drop, shun purge, horizon cost and cone preparation.
Zero, the default, keeps every value inline and creates no table, so local stores stay byte-identical. A store with blobpart rows is not readable by stock fossil; enable it only for hosted backends.
Closes #459 (user: [email protected], size: 1493) [annotate] [blame] [check-ins using]
// Package blob handles content-addressed blob storage in Fossil // repository databases. // // Fossil's blob format is a 4-byte big-endian uncompressed-size prefix // followed by zlib-compressed data. [Compress] and [Decompress] handle // this encoding transparently. // // [Store] compresses content, computes its SHA1 hash, and inserts it // into the blob table. [Load] retrieves and decompresses a blob by RID. // [StoreDeltaRaw] stores content that arrived over the wire already // delta-encoded, without expanding it. [EncodeForStorage] is the shared // decision point for whether a receive path can skip [Compress] entirely // and write already-encoded wire bytes straight to blob.content. // // A row's stored bytes are read and written only through [ReadStored], // [WriteStored], [InsertStored], [StoredHead] and [DropStored] (and the scan // expressions [StoredLenSQL] and [StoredPrefixSQL]), because a value over // db.DriverConfig.MaxValueBytes lives in blobpart rows, not blob.content. // // This package does not decide what gets delta-encoded. Deltifying an // artifact that is already stored means rewriting an existing row, not // inserting a new one, and the decision needs to expand content, which // this package cannot do without importing internal/content. Both live // in content.Deltify, which is the only place the policy is stated. // [StoreDelta] inserts a new row as a delta and has no callers outside // tests; it is not the commit path's primitive. package blob