| Comment: | Add a storage measurement mode to the bench harness (zig build bench -- storage): report bytes at rest by frame kind and artifact type, per-frame overhead, full-form compression headroom, and serve CPU, so storage-efficiency work is chosen on measured data rather than guesses. |
|---|---|
| Timelines: | family | ancestors | descendants | both | trunk |
| Files: | files | file ages | folders |
| SHA3-256: |
a48a0010ef4d9c12f70495c7f4772dc5 |
| User & Date: | dmestas on 2026-07-16 14:54:01.936 |
| Other Links: | manifest | tags |
|
2026-07-16
| ||
| 16:49 | Make zit pack monotonic so it never inflates a store: preserve each source frame verbatim (re-linking delta containers to their base's new rid) and re-deltify a full-form frame only when the result is strictly smaller, fixing a regression that discarded wire deltas and grew a fresh clone 8.65x. check-in: eb9a1de034 user: dmestas tags: trunk | |
| 16:40 | Make pack monotonic: preserve stored wire deltas verbatim instead of reconstructing every frame to full form, which inflated a delta-bearing store many times over closed check-in: 50a4b5d049 user: dmestas tags: packfix | |
| 16:15 | Add a manifest-transform projection to the bench harness (zig build bench -- storage --probe-manifest): read-only, it estimates the compressed win from replacing resolvable 40/64-hex manifest hashes with rid-ordinal varints, so survey 2.5 is decided on a real zlib delta rather than a guess. closed check-in: 64fd19938f user: dmestas tags: probeman | |
| 15:32 |
Add a read-only --probe-dict projection to the storage bench to gate dictionary-frame work.
Before building dictionary-compressed full-form frames, measure the win. The new 'zig build bench -- storage --probe-dict' mode (optional .zit path) samples a deterministic content-addressed dictionary from the corpus: up to 64 evenly spaced other-class artifacts plus 16 manifests, capped at 256 KiB. For every full-form zlib frame it then projects storing the artifact as a fossil delta against that dictionary: it deltas the content, verifies delta.apply reconstructs the bytes exactly, zlib-wraps the delta as a section 4.1 container, and keeps the projection only when it strictly beats the current frame. The one-time dictionary cost is included and every non-reconstructing or unprofitable frame is counted at its current size, so the number is honest, not a best case. It stays read-only: the store is opened through openReadOnly and never written. A clear PROCEED/DEFER line reports the projected reduction against a 15 percent gate. Technique after Yann Collet's zstd dictionary training (https://github.com/facebook/zstd) and the RFC 1950 section 2.2 FDICT preset- dictionary concept (https://www.rfc-editor.org/rfc/rfc1950), with results validated by Chrome's shared-dictionary transport deployment (https://developer.chrome.com/blog/shared-dictionary-compression). The matcher is Fossil's own delta format, so the projected bytes at rest stay a section 4.1 zlib container and no new codec is introduced. closed check-in: c9bea85ebc user: dmestas tags: probedict | |
| 14:54 | Add a storage measurement mode to the bench harness (zig build bench -- storage): report bytes at rest by frame kind and artifact type, per-frame overhead, full-form compression headroom, and serve CPU, so storage-efficiency work is chosen on measured data rather than guesses. check-in: a48a0010ef user: dmestas tags: trunk | |
| 14:51 | Add a storage bench mode reporting bytes at rest by frame-kind and artifact-type plus serve CPU, so the storage-efficiency ideas can be weighed against measured numbers before any claim. closed check-in: e782228b2d user: dmestas tags: sbench | |
| 14:25 | Add a zit pack command that repacks the store into a compact new-generation log: whole-frame-verify the source read-only, re-ingest every artifact in offset order re-deltaing full-form frames against lineage bases, then atomically swap via a fsynced temp — a crash before the rename leaves the original byte-identical, and it doubles as an integrity scrub. check-in: de49e67c9f user: dmestas tags: trunk | |