zit: zit 0.3.1

zit 0.3.1 is a point release with one change in it: cloning a large history is no longer slow. On the pkgsrc corpus — 305,369 check-ins over trees of about 100,000 files — zit clone of a flat store did not finish; it was killed after 3h52m. It now completes in under 20 minutes, and in zit's own sub-manifest format it lands within 14% of fossil clone of the same history while producing a store a third the size.

Nothing else changed. No artifact, tag namespace, or sync message differs from 0.3.0.

The change (zit#172)

A clone transfers artifacts and then derives the local caches, and the derive records, for every check-in, which files it changed against its parent. It did that by materializing the check-in's complete effective file list and its parent's, then diffing the two — work proportional to the size of the tree, paid once per check-in. On pkgsrc that is 100,000 entries twice over, 305,369 times, with every path and hash heap-allocated.

Fossil does not do this. Its add_mlink merges the two check-ins' own changed-file runs and seeks the baseline they share, which is work proportional to the change, not to the tree. zit now does the same:

Two delta manifests over one baseline diff over their own F cards, with a binary search into the baseline for a name only one side mentions.
Otherwise — a root check-in, a baseline change, a full manifest on either side — the two effective lists are walked side by side and compared as they are yielded. Neither list is ever materialized.
Recently read manifests stay parsed in a bounded cache (Fossil keeps six warm; this keeps 64), and an entry a capture step is using is pinned so it cannot be evicted under the step holding it.

The derived records are byte-identical to what the previous code produced: the .zit-cache files a clone writes compare equal before and after, and zit timeline -v over pkgsrc agrees with fossil timeline -v on every check-in and every file.

Measured

All figures on one machine, quiet, with fossil clone and zit clone interleaved so both see the same conditions. Fossil 2.28.

corpus	`fossil clone`	`zit clone`
sqlite (~30k check-ins)	96s	93s	parity
tcl (~25k check-ins)	391s	93s	zit 4.2x faster
pkgsrc, sub-manifest store	531s	603s	1.14x
pkgsrc, flat store	531s	1,063s	2.0x

Before this change the pkgsrc flat clone was killed at 13,963s without finishing, against 531s for fossil clone — better than 26x slower.

Two things the numbers say that the headline does not:

The remaining gap is the compatibility format, not zit. A flat store (import-fossil --compatible) preserves Fossil's artifacts verbatim, and those manifests are large full-text documents: 1,816 bytes per stored delta, 1.73GB for pkgsrc. The same history as sub-manifests is 186 bytes per delta and 663MB, and clones in 603s. Packing the flat store does not close this — zit pack on it costs 974s, improves the chains by 6.5%, and changes clone time by nothing measurable. The flat form cannot be packed into the sub-manifest form's shape; that is what conversion is for.

Timings on a loaded machine are worthless. Repeated runs of the same binary over the same input varied by 29% while other work shared the box. Every figure above was taken with the machine otherwise idle, and each was run twice.

Compatibility

Artifacts, tags, and the sync protocol are unchanged; Fossil RFC revision 00 remains the target. A stock fossil clones, pushes, and pulls a 0.3.1 repository byte-for-byte.
No on-disk record changed. A 0.3.0 store, checkout record, undo journal, and cache are read unchanged, and a 0.3.0 binary reads a 0.3.1 store.
No command, flag, or output changed.

Qualification

zig build test, e2e, smoke (Fossil interop against a real fossil), muscle (41 differential steps), git-import-e2e, git-export-e2e, and dst, plus the three-runner CI battery (macOS ARM64, Linux x86-64, Linux ARM64). Three tests are new: the delta-level diff against a hand-built baseline, its agreement with the walk-based diff, and a delta chain longer than the manifest cache, which exercises the eviction path a small fixture never reaches.