C Sixty-three\smegabytes\sto\ssay\snothing
D 2026-09-11T18:49:33.591
E 2026-09-11T13:40:00 b367f7a0a0de47bd709929c0b0bb01e5724392ff
N text/x-markdown
U dmestas
W 4862
# Sixty-three megabytes to say nothing
Two releases ago a no-op pull — both peers already holding everything —
took seventeen minutes. One release ago it took seven seconds. This one
looks at what those seven seconds were made of, and finds that most of the
bytes on the wire were a peer saying, one line at a time, "I have this" about
nine hundred thousand things the other side already knew it had.
That is what `igot` does. A pull opens with the client announcing every
artifact it holds, so the server can suppress what it need not send and can
ask for what it lacks. It is the right protocol for a repository of a
thousand artifacts. For pkgsrc it is 904,898 lines, about sixty-three
megabytes, per direction, per round — and every round of every pull, whether
or not anything changed. It fits under zit's reply ceiling today. At a few
million artifacts it would not, and the pull would fail outright, which is
the kind of limit that arrives as an outage rather than a slowdown.
Fossil solved this in 2007 with the cluster artifact. A cluster is the
simplest artifact in the format: a sorted list of hashes and a checksum,
nothing else. It is `igot` in bulk. Once a server folds an artifact into a
cluster it stops announcing that artifact individually — the cluster's M
cards are the announcement — and because a cluster is itself an artifact, a
later cluster names it, and the whole repository becomes reachable from the
newest one by following the chain down. A peer that already holds the chain
announces almost nothing; a peer that lacks it asks for one cluster and
learns about eight hundred artifacts at once.
zit could read clusters. It had never used them. On receipt it folded their
members into its request set correctly, but on send it announced everything,
and it never minted a cluster of its own. A repository born in zit had no
clusters at all, and a repository imported from fossil carried fossil's
clusters as dead weight — 2,439 of them in pkgsrc, naming nearly every
artifact, consulted by nothing.
This release consults them. A sync now keeps a small index — the names of
every held cluster, kept warm by the persist path as it writes and rebuilt by
one walk if lost — and resolves it at open into two facts: which artifacts
some cluster already names, and which names a held cluster promises that this
repository never collected. The first is what not to announce. The second is
the part fossil gets for free from its database and zit has to derive: fossil
records a phantom row for every promised-but-absent artifact and asks for it
on every sync until it arrives; zit writes no phantom to disk, so it plants
them again at each open from the clusters it holds. Without that, a pull
interrupted halfway would leave holes that no later pull could see, because
the bounded announcement is precisely the announcement that no longer
mentions them.
Then it announces held-and-unclustered, on both sides, and mints. When a pull
reply finds a hundred or more held artifacts that no cluster names, it folds
them into clusters of eight hundred — sorted, checksummed, stored like any
other artifact — exactly where fossil does it and at fossil's thresholds. A
repository born in zit clusters itself the first time anyone pulls from it.
Measured on the fossil-imported pkgsrc store, a no-op pull that sent
sixty-three megabytes now sends 4,748 bytes. Most of those are not
announcements at all: they are sixty-three `gimme` cards, for sixty-three
artifacts that pkgsrc's clusters promise and that no peer holds — private or
shunned on the fossil side, years ago. Fossil asks for those every round and
says nothing. zit's first version of this change did something worse: it
counted them as artifacts the peer had failed to deliver and reported every
pull as a failure, forever. The fix was already written down in the code, in
the comment explaining why a shunned phantom is not counted: an expectation
the peer set and did not meet is a failed pull; a request the run made on its
own initiative is not. Cluster promises are now a one-line warning, and the
pull says `already in sync`.
On a repository born in zit, one hundred and sixty-one artifacts, the first
pull after this release minted one cluster on the server and delivered it;
the next announced one artifact instead of a hundred and sixty-one — two
hundred and one bytes.
The index costs one walk of the store the first time a sync opens it —
three and a half minutes on pkgsrc, once, then a hundred-kilobyte file
adopted in a few milliseconds. And where the index is absent or stale, every
path degrades to the old behaviour: the full announcement, correct and
expensive. That was the design constraint throughout. Bounded announcement
trades bytes for bookkeeping, and the bookkeeping is allowed to be wrong only
in the direction that costs bytes.
Z d3b479a40f3d28f41811603a9c8797a1