Two fixes in 0.3.17, and each is a small lesson about where a cost or a bug actually lives versus where it first appears to.
An authenticated push that was never authenticated
0.3.16 made a saved login stand in for a push that named no user, so that
autosync would stop going out anonymous. Then a real fossil peer refused the
authenticated push anyway: "not authorized to write". The same login, on the
same peer, had just authenticated a config pull and been handed the user
table, which fossil serves to administrators only. Read accepted, write
refused, one credential. That asymmetry looked like a deep protocol question
about how fossil gates a multi-round write.
It was not. A logging proxy between zit and fossil showed the push request
carried no login card and no login cookie at all. It was anonymous on the
wire, and fossil was right to refuse it. The credential had been dropped
before the push was built: zit push --user U --password P with no URL,
relying on the saved remote, parsed to a "no URL" result that carried no
other fields, and the verb rebuilt its arguments from that as a bare empty
URL. sync never had the bug because its parser treats an omitted URL as an
empty one and keeps everything else. Every test I had written for the saved
login omitted the URL, so every one of them hit it, and I read the result as
fossil refusing rather than zit not asking.
The fix is a parser mirroring sync. The guard is the S12 interop scenario:
commit into a clone of a real fossil peer whose anonymous user cannot write,
push with a user and no URL, then push with no user at all, and assert both
land on the peer's timeline. The second push is the one 0.3.16 promised.
The pointer that got me there was libfossil having fixed its own sync-auth
bug the week before, which sent me to fossil's source instead of my theories.
fossil's check_login and its x-f-l-c cookie were exactly what zit
implements; the login was fine. The bug was upstream of all of it.
Reading a tree without reading it
0.3.16 made the file index work on a submanifest repository by flattening each tree check-in and its parent to complete file lists and diffing those. Correct, and 34 minutes once on a pkgsrc mirror of 218,444 check-ins, because the cost was files-per-check-in — 68,588 — not files-changed.
A tree is a list of S cards, each naming a sub-manifest by hash, and a check-in shares almost every sub-manifest with its parent by the premise of the format. So the diff now walks the two S-card runs side by side. Where the hash is the same the whole subtree is skipped and neither side is even read. Only where it differs are both sides resolved, their files diffed, and the walk recursed. On pkgsrc that is about four sub-manifests read per check-in against every one before, and the build fell from 34 minutes to under seven. Answers were checked per path against the flat twin and against reconstruction; nothing moved.
One subtlety worth recording. A partition's key is the smallest filename in
it, so removing a file can move the key, and the same live file then sits
under an old key on the parent side and a new key on the child side. Walked
in key order, the old key can come first, and a delete recorded then would
make a file the child still holds look gone. The index would not care — a row
is a row — but annotate reads the disposition unconfirmed. Deletes are
therefore held until the whole diff has run, and only a file the child truly
lacks is recorded as one. The reviewer caught this; the fixture I had written
never moved a key.
The other thing 0.3.16 got slightly wrong is undone here too: it flattened trees eagerly when the diff memo parsed them, and that memo also serves the derive feed on every cold open. A flat child with a tree parent could flatten that parent on open. Trees are now flattened only on demand, only in the index build, only for the pairings that are full enumerations by nature.