50 descendants of 997edb13b538fef3
|
2026-07-16
| ||
| 22:16 | Add a read-lenient import path so zit can read fossil's own pre-2009 history: parseForImport tolerates the single legacy trailing space on fossil F cards (a cvs2fossil quirk, non-conforming per artifact-format 4.2) by stripping one dangling separator per F card ONLY after the strict parse rejects and the original Z self-checksum validates, re-sealing Z on an ephemeral copy that never touches storage, serve, or identity. The strict parser, the writer, and the RFC stay unchanged; artifacts keep their original hash. closed check-in: cca7dc06a2 user: dmestas tags: fcard-import | |
| 21:39 | Add a zit cat verb emitting one file's uncompressed content at a resolved check-in through the Repository facade, per artifact-format §8.4 single-name lookup and §7.2 hash-verified blob reconstruction check-in: d0ef5067bb user: dmestas tags: cli | |
| 21:38 | Create new branch named "fcard-import" check-in: d3be8fe4ac user: dmestas tags: fcard-import | |
| 21:13 | Create new branch named "fcard-fix" closed check-in: 65813afbe9 user: dmestas tags: fcard-fix | |
| 21:10 | Merge stream-clone: cut clone peak memory 14.2 GB -> 291 MB (fossil parity) by storing received frames wire-backed instead of buffering the whole repo's reconstructed plaintext, with section 7.7 verification moved onto the log-basis reconstruction path (a stronger round-trip parity guard). check-in: 77e2541742 user: dmestas tags: trunk | |
| 21:08 | Stop buffering the whole repository's reconstructed plaintext in RAM during clone: store received frames wire-backed and verify each durable frame's section 7.7 hash against its log-basis reconstruction (sync 7.2/7.4), cutting a sqlite clone's peak memory from 14.2 GB to 291 MB with a byte-identical store and a matching trunk hash. closed check-in: 9a18bdd63d user: dmestas tags: stream-clone | |
| 21:06 | Add a zit ls verb listing a check-in's effective file set through the Repository facade, per artifact-format §8.2 file enumeration check-in: 6ee0c0e855 user: dmestas tags: cli | |
| 20:39 | Merge tcard-order: fix T-card run ordering to key on the full sign-included name token (artifact-format 4.7.16), clearing 159 wrongly-rejected branch-switch check-ins that broke zit log on deep histories. check-in: b4f2ae0ceb user: dmestas tags: trunk | |
| 20:37 | Order T-card runs by the full sign-included name token per artifact-format 4.7.16, not the sign-stripped name, so branch-switch check-ins with interleaved propagate/cancel tags (T *sym-trunk / T -sym-clear-title) parse instead of being wrongly rejected as out-of-order. closed check-in: a2fa7629c6 user: dmestas tags: tcard-order | |
| 20:27 | Fold the store layers and RepoView behind one zit.Repository facade so read verbs open a repo, resolve names, and read content through a single deep interface check-in: 513397e07a user: dmestas tags: cli | |
| 20:15 | Create new branch named "tcard-order" check-in: 83a1201013 user: dmestas tags: tcard-order | |
| 20:06 | Create new branch named "stream-clone" check-in: 14f733376d user: dmestas tags: stream-clone | |
| 19:14 | Merge trunk onto cli: bring the branch tip up to the verbatim-serve, delta-cfile, repostate cache, container codec, and storage-benchmark work before starting the CLI facade milestones. check-in: a81d55773e user: dmestas tags: cli | |
| 16:49 | Make zit pack monotonic so it never inflates a store: preserve each source frame verbatim (re-linking delta containers to their base's new rid) and re-deltify a full-form frame only when the result is strictly smaller, fixing a regression that discarded wire deltas and grew a fresh clone 8.65x. check-in: eb9a1de034 user: dmestas tags: trunk | |
| 16:40 | Make pack monotonic: preserve stored wire deltas verbatim instead of reconstructing every frame to full form, which inflated a delta-bearing store many times over closed check-in: 50a4b5d049 user: dmestas tags: packfix | |
| 16:15 | Add a manifest-transform projection to the bench harness (zig build bench -- storage --probe-manifest): read-only, it estimates the compressed win from replacing resolvable 40/64-hex manifest hashes with rid-ordinal varints, so survey 2.5 is decided on a real zlib delta rather than a guess. closed check-in: 64fd19938f user: dmestas tags: probeman | |
| 15:32 |
Add a read-only --probe-dict projection to the storage bench to gate dictionary-frame work.
Before building dictionary-compressed full-form frames, measure the win. The new 'zig build bench -- storage --probe-dict' mode (optional .zit path) samples a deterministic content-addressed dictionary from the corpus: up to 64 evenly spaced other-class artifacts plus 16 manifests, capped at 256 KiB. For every full-form zlib frame it then projects storing the artifact as a fossil delta against that dictionary: it deltas the content, verifies delta.apply reconstructs the bytes exactly, zlib-wraps the delta as a section 4.1 container, and keeps the projection only when it strictly beats the current frame. The one-time dictionary cost is included and every non-reconstructing or unprofitable frame is counted at its current size, so the number is honest, not a best case. It stays read-only: the store is opened through openReadOnly and never written. A clear PROCEED/DEFER line reports the projected reduction against a 15 percent gate. Technique after Yann Collet's zstd dictionary training (https://github.com/facebook/zstd) and the RFC 1950 section 2.2 FDICT preset- dictionary concept (https://www.rfc-editor.org/rfc/rfc1950), with results validated by Chrome's shared-dictionary transport deployment (https://developer.chrome.com/blog/shared-dictionary-compression). The matcher is Fossil's own delta format, so the projected bytes at rest stay a section 4.1 zlib container and no new codec is introduced. closed check-in: c9bea85ebc user: dmestas tags: probedict | |
| 14:54 | Add a storage measurement mode to the bench harness (zig build bench -- storage): report bytes at rest by frame kind and artifact type, per-frame overhead, full-form compression headroom, and serve CPU, so storage-efficiency work is chosen on measured data rather than guesses. check-in: a48a0010ef user: dmestas tags: trunk | |
| 14:51 | Add a storage bench mode reporting bytes at rest by frame-kind and artifact-type plus serve CPU, so the storage-efficiency ideas can be weighed against measured numbers before any claim. closed check-in: e782228b2d user: dmestas tags: sbench | |
| 14:25 | Add a zit pack command that repacks the store into a compact new-generation log: whole-frame-verify the source read-only, re-ingest every artifact in offset order re-deltaing full-form frames against lineage bases, then atomically swap via a fsynced temp — a crash before the rename leaves the original byte-identical, and it doubles as an integrity scrub. check-in: de49e67c9f user: dmestas tags: trunk | |
| 14:16 | Add a zit pack verb that repacks the store into a compact new generation: re-ingest every artifact in offset order through a parent-manifest and same-path-blob lineage hint so full-form frames re-delta against their basis, recompute each reconstruction's hash against its name before writing it (repo-state §7.7), and swap the rewritten log in with an atomic rename so a crash before the swap leaves the original store byte-identical. closed check-in: 43dccab01c user: dmestas tags: pack | |
| 13:38 | Store cloned artifacts as compact delta frames: retain fossil's reverse-delta containers through the dangling queue and persist each basis before the delta that points at it, so a clone from a fossil origin keeps the server's deltas instead of re-inflating every artifact to full form — cutting on-disk size to roughly 0.6x of the full-form store, matching fossil's own delta count. check-in: 9853f8b93e user: dmestas tags: trunk | |
| 13:28 | Keep fossil's reverse-delta containers through the dangling queue and persist each basis frame before the delta that points at it, so a clone from a fossil origin stores compact delta frames instead of re-inflating every artifact to full form. closed check-in: 0ebc7991f0 user: dmestas tags: storage | |
| 13:26 | Merge the sync-protocol re-sync onto trunk: ratify the container-capacity and clone-igot clarifications that match the reference implementation and zit's own behavior. check-in: 61fde8dec1 user: dmestas tags: trunk | |
| 13:23 | Re-sync the sync-protocol draft: the container length field is a receiver capacity, so a decoded stream shorter than the advertised value must not be rejected (4.1), and igot is never applicable during clone rounds (8.2 / Table 2) — both aligning the text with the reference implementation. closed check-in: d708b4a2f2 user: dmestas tags: rfc-resync | |
| 11:26 | Stop the clone client re-announcing every held artifact with igot each round: a clone enables no push direction (sync 8.2/8.5) so the server ignores those cards, and dropping them cuts a 50k-artifact clone request from ~88 MB to under 2 KB while convergence still rides the server's RID cursor. check-in: 289e073d9c user: dmestas tags: trunk | |
| 11:16 | Stop the clone client re-announcing every held artifact with igot each round: a clone enables no push direction (RFC 8.2/8.5) so the server ignores those cards, and dropping them cuts a 50k-artifact clone request from ~88 MB to under 2 KB. closed check-in: ebc8401895 user: dmestas tags: igot | |
| 10:33 | Wire zit_storage and zit_logstore into the fuzz executable so the seeded fuzz suite compiles again: its sim helpers and logstore targets import those modules, which every other build artifact already provides but the fuzz exe was missing. check-in: 490977ea08 user: dmestas tags: trunk | |
| 10:25 | Wire zit_storage and zit_logstore into the fuzz executable so the seeded fuzz suite compiles again: its sim helpers and logstore targets import those modules, which every other build artifact already provides but the fuzz exe was missing. closed check-in: 1c7715c645 user: dmestas tags: fuzzbuild | |
| 09:58 | Add --no-cache to the read-only repo-state verbs (leaves, branch, info, tag): bypass the .zit-cache memo and force a full-log re-derive, giving a first-class integrity/debug escape hatch that re-verifies the whole log instead of trusting the O(1) tail check. check-in: 5af2298bc9 user: dmestas tags: trunk | |
| 09:55 | Add --no-cache to the read-only repo-state verbs to bypass the persisted cache and force a full-log re-derive that re-verifies whole-log integrity closed check-in: 6e3529ad92 user: dmestas tags: nocache | |
| 09:40 | Add a persisted derivation cache so repo-state verbs skip re-deriving all of history: cache the crosslink inputs keyed on the append-only log's tail and replay them on a hit, cutting a warm 'zit leaves' on a 130k-artifact repo from ~103s to 0.21s with byte-identical output and a full rebuild on any log change or corruption. check-in: e1f401fa0d user: dmestas tags: trunk | |
| 09:38 | Document that a cache hit skips whole-log integrity, so interior bitrot preserving size and tail is served until a rebuild — deleting .zit-cache restores full detection. closed check-in: 31e140ddbc user: dmestas tags: cache | |
| 09:28 | Memoize the derived repo view in a disposable .zit-cache sidecar so read-only verbs answer sub-second at fossil scale instead of rescanning the whole log on every invocation. check-in: 08504d5f5d user: dmestas tags: cache | |
| 07:28 | Extract the sync 4.1 zlib container codec into one leaf module: the three copies had drifted — logstore and repostate rejected valid containers whose zlib stream decodes to fewer bytes than the declared capacity, stricter than 4.1's output-capacity semantics — so unify on the spec-correct decode and retire the duplication. check-in: 0b8ff7a1f8 user: dmestas tags: trunk | |
| 07:23 | Extract the §4.1 zlib container codec into one leaf module (src/container.zig) so logstore, repostate, and sync share a single §4.1-capacity-semantics implementation instead of three drifting copies. closed check-in: 0e5cc09150 user: dmestas tags: container | |
| 07:05 | Retain received cfiles verbatim on the clone client: store each artifact's received container or delta as-is after verifying its hash over the reconstructed content, so delta compactness survives clone-of-clone and client disk roughly halves — and reject a lying-named push the server previously accepted. check-in: a5ef5d18c0 user: dmestas tags: trunk | |
| 06:53 | Retain received cfile containers and deltas verbatim on the clone client so delta compactness survives a clone-of-clone, verifying each reconstructed artifact's hash at the persist commit hook before storing the wire form rather than re-expanding every received delta to full form. closed check-in: 5526cf3536 user: dmestas tags: d4 | |
| 06:13 | Speed up repo-state derivation on open: replace the O(N-squared) crosslink scans with parent/child/tag adjacency indexes, verify each frame's whole-frame CRC once at scan instead of twice per verb, and cache delta bases during the derivation sweep — cutting the algorithmic crosslink term over 100x with byte-identical derived state. check-in: baa7053ee1 user: dmestas tags: trunk | |
| 06:11 | Assert a read-only-verified store is never appended to: the append frame path skips the redundant whole-frame CRC only because openReadOnly proved it at scan, so appending to such a store would leave a frame unchecked. closed check-in: 7c3d36865e user: dmestas tags: scale | |
| 06:00 | Accelerate repo-state derivation on open: replace the quadratic crosslink scans with per-parent/child/rid adjacency indexes, verify each frame's CRC once at open instead of again on every read, and memoize delta bases across the derivation sweep, so the query verbs stay fast on large repositories without changing any derived answer. check-in: 76b0f8f1d0 user: dmestas tags: scale | |
| 05:04 | Delta-encode cfiles on clone: store an artifact as a delta against its previous version when materially smaller, and serve that stored delta container verbatim with the reconstructed-size USIZE per sync 7.2, cutting clone reply size 58-87% on version-chained repos while full-form artifacts still serve unchanged. check-in: d7e16fadb1 user: dmestas tags: trunk | |
| 05:01 | Drop the unused putContainer store method and its two tests: its only intended caller is the deferred client-verbatim persist hook, so it stays dead surface until that lands. closed check-in: 673d5070bb user: dmestas tags: delta | |
| 04:50 | Use the known reconstructed size for a delta corpus artifact's USIZE instead of re-parsing the delta header: the bench just created the delta from cur, so its reconstructed target is cur and its length is the section 7.2 USIZE directly, dropping a redundant header parse. check-in: 3bdb4f96c0 user: dmestas tags: delta | |
| 04:47 | Delta-encode blobs and manifests on ingest and serve them verbatim as section 7.2 delta cfiles on clone: store a commit's blob (against the same path's parent blob) and manifest (against the primary parent) as a delta when it comes in materially smaller and within the write-depth cap, carry the reconstructed-target size from the delta header as the cfile USIZE rather than the container's raw-delta-length prefix, and retransmit the stored delta container byte-for-byte so a chained-corpus clone reply drops 58-87 percent while a full-form corpus is byte-identical to before. check-in: 71235bf3be user: dmestas tags: delta | |
| 03:57 | Serve stored cfile containers verbatim on clone: reuse each artifact's already-compressed 4.1 container instead of recompressing it on every clone reply, moving the per-clone compression cost off the serve path to ingest and cutting clone-serve time ~76% with byte-identical wire output. check-in: f4fe62b160 user: dmestas tags: trunk | |
| 03:51 | Restrict verbatim cfile serving to the full form: the delta serve path had no production writer and wrote the delta's length as USIZE instead of the reconstructed artifact size (sync §7.2), so drop it and let delta-stored artifacts fall back to a conformant reconstructed full-form cfile. closed check-in: 8e430e7bd7 user: dmestas tags: perf | |
| 03:24 | Serve stored cfile containers verbatim on clone, reusing each artifact's already-compressed container instead of recompressing every clone reply so the per-clone compression cost moves off the serve path to ingest. check-in: 84d47c3a21 user: dmestas tags: perf | |
| 02:41 | Land the out-of-band sync clone benchmark on trunk: it splits clone time into build/frame/decode/respond/reframe/consume/persist/reopen phases behind a convergence-verified in-process clone, and attributes per-artifact cfile compression as a faithful share of the server respond phase, so the first clone-speed optimization targets measured work rather than a guess. check-in: 5f6f1286d1 user: dmestas tags: trunk | |
| 02:35 | Report the sync clone benchmark's server phase honestly: drop the unmeasurable walk+compress vs decode+parse split and show only the standalone compression micro as a share of respond, since scanRequest and emitCloneReply are private so parse and walk can't be told apart at the public boundary. closed check-in: 53962101c0 user: dmestas tags: bench | |
| 02:10 | Split the sync benchmark's server phase into timed decode, respond and reframe steps plus a standalone cfile-compression measurement, so clone-speed work can tell inbound igot decode+parse apart from the actual per-artifact compression instead of trusting one opaque server label. check-in: 997edb13b5 user: dmestas tags: bench | |