A pull into an already-up-to-date clone of the Fossil SCM repository took 2m44s and moved nothing. A CPU profile of that no-op pull puts 134.4s of 149.1s of samples (90%) inside manifest.CrosslinkContext, the whole-repository sweep Sync runs after the round loop. Nothing in the protocol path registers.

Two independent costs, both O(repository size) and neither bounded by what arrived:

collectCrosslinkCandidates 99.5s (67%) linkBatch -> Cache.Expand 34.9s (23%) of which hash.SHA3 26.0s (17%)

The query was quadratic. Its four exclusions were correlated NOT EXISTS subqueries, and tagxref is indexed on (rid, tagid) and (tagid, mtime) -- neither covers srcid -- so SQLite rescanned all 51,134 tagxref rows once per blob, 67,615 times. EXPLAIN QUERY PLAN showed the CORRELATED SCALAR SUBQUERY -> SCAN tx. Rewritten as NOT IN, each set materializes once and is probed: identical 39,727 rows, 47.7s -> 0.08s measured under canonical fossil's SQLite.

The candidate set never shrank. A blob qualifies when it has none of the derived rows a crosslink writes, and ordinary file content never gains those rows however often it is examined -- so 39,727 of the repository's 67,615 blobs were re-expanded (delta chain walked, SHA3 re-verified) and re-parsed on every sync. A probe run confirms the sweep linked exactly 0 artifacts on each of two consecutive runs over the converged repository, at 3m20s and 3m15s.

Canonical fossil has no equivalent cost because it never sweeps: xfer.c attempts one crosslink per blob as it arrives, artifact or not. This gives the sweep the same once-per-blob property via crosslink_nonartifact, an on-demand table recording blobs that expanded but hold no parseable artifact. Both facts are properties of immutable blob content, so the verdict cannot go stale; a blob that failed to expand, or that parsed but was deferred, is deliberately not recorded and is reconsidered next sweep.

Measured on a clone of the Fossil SCM repository (67,615 blobs), no-op pull:

before 2m43.9s after 0.40s

Clone is unchanged at 5m30s (it pays the one-time examination either way), and the two clones' derived tables are identical -- blob, event, mlink, plink, tagxref, filename and leaf totals all match row for row.

Fixes #202 check-in: 3748b95de7 user: [email protected] tags: trunk