| Comment: | storage: a block cache for FileStorage reads, opt-in and not yet wired
Profiling a real pkgsrc-scale import put readv at over half of all CPU samples. A block cache in front of pread is the shape that addresses it: the store is append-only, so a block is retained only when the read that filled it came back FULL -- which proves those bytes already existed, so appends cannot alter them and need no invalidation. Only truncate does. Deliberately not attached to any caller. Wired into import-git it reached a 99 percent hit rate over 392,421 reads and did NOT make a small corpus faster: at that size the page cache already served those reads, so the syscalls were cheap and the lookup was pure overhead. The prior from the real profile is strong but unproven at scale, and shipping it on 'should help' is how the reverted top-level cache in issue 32 happened. useReadCache is the one-line opt-in for profiling it. (#39) |
|---|---|
| Timelines: | family | ancestors | descendants | both | sub-manifest |
| Files: | files | file ages | folders |
| SHA3-256: |
f3becbedd1ee8df30506ce89788f2b52 |
| User & Date: | dmestas on 2026-08-07 13:03:45.705 |
| Other Links: | branch diff | manifest | tags |
|
2026-08-07
| ||
| 15:29 |
import-git: name the main line trunk, whatever git called it
A git default branch is master or main; this format's is trunk, and so is every place that must name the main line without being told -- a fresh checkout, zit open resolving a tip, a clone initialising its own state. A repository whose main line was called master satisfied none of them: it cloned and verified clean, then could not be opened, because the branch the destination looked for held no commits. labelBranches now takes the name to MATCH on and the name to APPLY separately: the head is still found by its source name, the segment is labelled trunk. Side branches keep their source names, and the git-side refs mapping still records sym-heads/master, so nothing about round-trip identity moves. If a side branch already publishes trunk, the source name is kept rather than colliding two segments under one label. (#42) check-in: 5caf866d86 user: dmestas tags: sub-manifest | |
| 13:03 |
storage: a block cache for FileStorage reads, opt-in and not yet wired
Profiling a real pkgsrc-scale import put readv at over half of all CPU samples. A block cache in front of pread is the shape that addresses it: the store is append-only, so a block is retained only when the read that filled it came back FULL -- which proves those bytes already existed, so appends cannot alter them and need no invalidation. Only truncate does. Deliberately not attached to any caller. Wired into import-git it reached a 99 percent hit rate over 392,421 reads and did NOT make a small corpus faster: at that size the page cache already served those reads, so the syscalls were cheap and the lookup was pure overhead. The prior from the real profile is strong but unproven at scale, and shipping it on 'should help' is how the reverted top-level cache in issue 32 happened. useReadCache is the one-line opt-in for profiling it. (#39) check-in: f3becbedd1 user: dmestas tags: sub-manifest | |
| 11:34 |
pack cache: name WHICH budget forced each eviction
A real 53,773-commit run reported 99,097 evictions against 69,485 lookups -- evicting more than it was asked about. But two caps can force that, max_bytes and max_entries, and they bind under different data, so the count alone said only 'raise something'. Evictions are now split by cause and the resident set is reported, so a run names the budget to raise instead of requiring a blind sweep. (#40) check-in: 88ae8ba619 user: dmestas tags: sub-manifest | |