Timeline

'repository is corrupt' says a repository cannot be read, not what in it is unreadable -- and the walk that discovers it visits every artifact in the store, minutes on a large one. Whoever holds the broken repository was left bisecting it or attaching a debugger.

The derive now records which artifact it rejected and why; openRepo reports it, so every verb's open gets the name. Plain data, no allocation, so it survives the errdefer that tears down a half-built view. Repository.open clears it per attempt because callers declare a Repository undefined.

Also splits open's two identical diagnostics: deriving the view and reopening the log for materialization both printed the same line, which made a failure in one indistinguishable from the other. (#42) check-in: 37f746e59f user: dmestas tags: sub-manifest

A real 53,773-commit import evicted 99,097 times, every one forced by the 4096-entry cap and not one by the 64 MiB byte cap, with the resident set at 23,551 KiB. The cache was held to a third of its own stated size by a second, unrelated number, and the 51 percent hit rate that produced looked like a working-set problem rather than an arithmetic accident.

The entry cap now divides the byte budget by a planning entry size, so the two cannot disagree about which one is bounding: bytes bind for normal data, entries only for many tiny items, which is what an entry cap is for. (#40) check-in: ac89bad5fd user: dmestas tags: sub-manifest

A git default branch is master or main; this format's is trunk, and so is every place that must name the main line without being told -- a fresh checkout, zit open resolving a tip, a clone initialising its own state. A repository whose main line was called master satisfied none of them: it cloned and verified clean, then could not be opened, because the branch the destination looked for held no commits.

labelBranches now takes the name to MATCH on and the name to APPLY separately: the head is still found by its source name, the segment is labelled trunk. Side branches keep their source names, and the git-side refs mapping still records sym-heads/master, so nothing about round-trip identity moves.

If a side branch already publishes trunk, the source name is kept rather than colliding two segments under one label. (#42) check-in: 5caf866d86 user: dmestas tags: sub-manifest

Profiling a real pkgsrc-scale import put readv at over half of all CPU samples. A block cache in front of pread is the shape that addresses it: the store is append-only, so a block is retained only when the read that filled it came back FULL -- which proves those bytes already existed, so appends cannot alter them and need no invalidation. Only truncate does.

Deliberately not attached to any caller. Wired into import-git it reached a 99 percent hit rate over 392,421 reads and did NOT make a small corpus faster: at that size the page cache already served those reads, so the syscalls were cheap and the lookup was pure overhead. The prior from the real profile is strong but unproven at scale, and shipping it on 'should help' is how the reverted top-level cache in issue 32 happened.

useReadCache is the one-line opt-in for profiling it. (#39) check-in: f3becbedd1 user: dmestas tags: sub-manifest

A real 53,773-commit run reported 99,097 evictions against 69,485 lookups -- evicting more than it was asked about. But two caps can force that, max_bytes and max_entries, and they bind under different data, so the count alone said only 'raise something'.

Evictions are now split by cause and the resident set is reported, so a run names the budget to raise instead of requiring a blind sweep. (#40) check-in: 88ae8ba619 user: dmestas tags: sub-manifest

PackStats carries the cache's hits, misses, evictions and oversized skips, and import-git prints them. Same reason SubManifestMemo got this in #35: a fixed budget that is wrong for a corpus is invisible from outside the process, and a hit count without its denominator cannot show it. Nothing here gates a decision. (#40) check-in: b6cd09fc11 user: dmestas tags: sub-manifest

The trailer is the four bytes immediately after the payload and was read by its own pread. FileStorage is unbuffered, so that was a second kernel transition for adjacent bytes, paid per frame -- and a deltified artifact walks its whole chain frame by frame.

Also gives PlaintextCache the counters it was missing. It tracked hits with no denominator, so a hit COUNT could not be read as a hit RATE; misses, evicted and oversized make the budgets observable. (#39, #40) check-in: a4f298d3e1 user: dmestas tags: sub-manifest

max_entries was 1 shifted by 20 while its own doc named the u32 count field that carries it, so insert refused a correct set at 1,048,576 paths -- four thousand times below what the format can express. Now the u32 range, with max_file_bytes computed from it rather than a flat 64 MiB, and read sized from stat like every other whole-file read here.

Two dependents moved in the same change, which is the point:

extras borrowed max_entries as a directory-sweep ceiling. Never the same quantity, and a tracked-set format change would have silently widened a filesystem runaway guard. It has its own now.

undo parseSlot allocated count entries BEFORE parsing them, guarded by count over max_files. That guard became vacuous the moment max_files reached the u32 range, leaving a 4-billion-entry claim reaching the allocator. Now bounded by what the remaining payload can back. (#38) check-in: 93adddd49d user: dmestas tags: sub-manifest

The ceiling was a PRODUCT of round cap and batch size, so shrinking the batch to 1 reaches it at a corpus a unit test can hold: 1124 artifacts, 1124+ rounds, stall budget 4.

Records that a converging clone ends exactly one stall deep -- the terminal round carries no cfile -- so the budget can never be 1, and that the count does not grow with the corpus. (#37) check-in: 5aa8e46e47 user: dmestas tags: sub-manifest

The cap was justified as bounding 'a lying on-disk size before any allocation'. The read path takes stat's size and allocates exactly that, so there was no claim to distrust. What it did do was silently refuse a larger cache -- and a cache is advisory, so that refusal is invisible, degrading to a full rebuild with no diagnostic.

Real bounds stay where they bind: the usize cast on read, the u32 image_len field on write. (#38) check-in: c82c04b10d user: dmestas tags: sub-manifest