Skip to content
zit Timeline
Login

Timeline

2 check-ins related to "probedict"

2026-07-16
15:32
Add a read-only --probe-dict projection to the storage bench to gate dictionary-frame work.

Before building dictionary-compressed full-form frames, measure the win. The new 'zig build bench -- storage --probe-dict' mode (optional .zit path) samples a deterministic content-addressed dictionary from the corpus: up to 64 evenly spaced other-class artifacts plus 16 manifests, capped at 256 KiB. For every full-form zlib frame it then projects storing the artifact as a fossil delta against that dictionary: it deltas the content, verifies delta.apply reconstructs the bytes exactly, zlib-wraps the delta as a section 4.1 container, and keeps the projection only when it strictly beats the current frame. The one-time dictionary cost is included and every non-reconstructing or unprofitable frame is counted at its current size, so the number is honest, not a best case. It stays read-only: the store is opened through openReadOnly and never written. A clear PROCEED/DEFER line reports the projected reduction against a 15 percent gate.

Technique after Yann Collet's zstd dictionary training (https://github.com/facebook/zstd) and the RFC 1950 section 2.2 FDICT preset- dictionary concept (https://www.rfc-editor.org/rfc/rfc1950), with results validated by Chrome's shared-dictionary transport deployment (https://developer.chrome.com/blog/shared-dictionary-compression). The matcher is Fossil's own delta format, so the projected bytes at rest stay a section 4.1 zlib container and no new codec is introduced. closed check-in: c9bea85ebc user: dmestas tags: probedict

14:54
Add a storage measurement mode to the bench harness (zig build bench -- storage): report bytes at rest by frame kind and artifact type, per-frame overhead, full-form compression headroom, and serve CPU, so storage-efficiency work is chosen on measured data rather than guesses. check-in: a48a0010ef user: dmestas tags: trunk