Until 0.3.9, zit clone held every artifact it received in memory until the
exchange converged, and only then wrote the repository. Correct, and the
simplest thing that could work — but its peak memory tracked the repository's
size, so a repository larger than RAM could not be cloned at all, and the
failure arrived after the whole transfer had been paid for.
The obstacle to writing earlier was ordering. Fossil stores an older artifact as a delta of a newer one, so a delta routinely arrives before the basis it names, while zit's log records a delta's basis as a backward offset — the basis frame must physically precede the delta. The batched pass resolved this with a worklist over the converged store; converged was the operative word.
0.3.9 runs that worklist per round instead, the shape of git's index-pack:
stream to disk, defer a delta whose base is unseen, resolve the moment it
lands. Two corners only show up mid-exchange and were caught by the new tests
before shipping. A delta whose basis has not arrived (as opposed to not yet
persisted) must defer rather than fall back to a full frame, or the streamed
repository's bytes differ from the batched one's. And fossil interns a
delta's unseen DELTASRC as a phantom the moment the delta arrives — the
content lands at that same record later, behind the drain's cursor, so the
pass keeps a revisit list for records it passed while they were empty.
Measured before built: an evidence seam (ZIT_PERSIST_WAITING=1) meters the
deferred set the drain would keep resident. Against a zit peer it is zero at
every scale — the log serves basis before delta by construction. Against
fossil 2.28 on the sqlite corpus it peaks near 11 MB: half the deltas wait,
but what waits is small wire payloads. On those numbers the spill-to-disk
fallback we had sketched was declined; one mechanism suffices.
On the pkgsrc corpus — 904,898 artifacts, a 1.73 GB store — the whole
repository is on disk with the clone process near 820 MB, where it previously
held everything. The streamed and batched passes write byte-identical
repositories, pinned by test, and every qualification clone finishes with a
clean zit verify. The verb's overall peak now belongs to the post-transfer
cache derive, which is its own problem (zit#182) with its own shape.
Cone-scoped clones keep the batched pass: their keep set is classified over the converged store, as the sparse-cone draft requires.