C A\sno-op\spull\sthat\stook\sseventeen\sminutes
D 2026-09-10T23:48:59.937
E 2026-09-10T18:50:00 6075779101cf1c36903410b5ab46ec3d473f69fe
N text/x-markdown
U dmestas
W 3582
# A no-op pull that took seventeen minutes
Every read verb in zit was measured and level with fossil. Clone was measured
and level. So when the question came up — should zit mint cluster artifacts
during sync, the way fossil does — the honest answer started with a smaller
one: what does zit actually cost on a pull today? The A/B table did not say,
because pull had never been in it. Clone was, and clone is exempt from the
announcement path by design; it streams by record cursor and announces
nothing. The one verb the table covered near this code was the one verb that
does not touch it.
The test was a pull where both peers already hold the entire repository. It
transfers nothing — it should be the cheapest sync there is. At 131k artifacts
it took ten seconds. At 1.25M it took seventeen and a half minutes, and did
nothing.
A stack sample of the server during that pull put 3,748 of 3,749 main-thread
samples in one place. When a peer announces `igot HASH` for something it holds,
the server should not turn around and announce the same hash back. So before
advertising each of its held artifacts, the server checked whether the peer had
already named it — by scanning the peer's announced list. Linearly. Once per
held artifact. The peer, doing an ordinary pull, had announced its whole
holdings, so both lists were the size of the repository, and the check was
quadratic: nine and a half times the artifacts, seventy-eight times the server
CPU. Not slower. It does not finish.
The fix is the boring one. Build the announced and requested names into a set
once per reply, and probe the set. The scan that was O(held × announced) is now
O(held). On the same 1.25M no-op pull the server's share of the work falls from
760 seconds to 4.9. The whole pull falls from 17.4 minutes to 4.7.
Four minutes is not fast, and the technote would be dishonest to stop there.
The quadratic is gone; what is left is linear, and all of it is one shape — to
announce its holdings the client materialized the whole repository, and now it
walks all of it again to persist what it received, which was nothing. The
server's 4.9 seconds is the real cost of the operation. The remaining four
minutes is the price of saying, one artifact at a time, "I have this" about a
million things. That is the next change, not this one.
What is worth keeping is not the fix but why it hid. The corpora the smoke and
muscle suites run on are small enough that a quadratic with a small constant
reads as zero. Every large-repository test in the project's history was a clone,
and clone was the one operation that never announces. A cliff can sit directly
under a green test battery for as long as no test stands where it is. So the
pull now has an evidence counter — announcement probes per reply — and a pin
that runs it at two repository sizes and requires the count to track the
holdings, not the announcement. A quadratic cannot pass that whatever the
absolute numbers are, and a both-sides-full pull is now part of what "qualified"
means.
The larger half of the problem is still open and now has a number under it: even
with the scan gone, a pull still announces every held artifact every round —
1.25M igot lines, about 87 megabytes per direction. It fits under the wire cap
today and fails outright past it. That is what cluster artifacts are for, and
whether zit should mint them is the question this whole investigation started
from. The answer is yes, and it is tracked as its own change — but first the
thing that turned a ten-second operation into a quarter hour had to go.
Z e816845cba455ce73a05286e2213c186