Parents and children of check-in [b82aacc805]
|
2026-08-24
| ||
| 18:33 |
logstore: index v3 carries a sorted directory, so opening builds no hash map
Adopting an index inserted every entry into a StringHashMap before answering anything: 255,167 keys on a 302MB store, to serve the one or two lookups a tree listing makes. Measured at 5.5ms of a 24ms open by building the map twice and taking the delta. v3 appends entry_count little-endian u32 rids ordered by digest, and lookup binary-searches it -- ~18 comparisons against a quarter-million insertions. Opening tcl drops 24.0ms to 20.0ms; with the hardware crc32 change before it, 38.5ms to 20.0ms. Output byte-identical. The directory carries RIDS and the entry records stay in scan order. Sorting the entries themselves would renumber every rid, and ancestry, phantom snapshots and the sync walk all read that numbering. lookup consults the map first, then the directory: entries appended after adoption -- a tail resume past the index boundary, and every write on a scan-path store -- are only in the map. Every held-check that was reading index.get directly now goes through lookup, because intern runs during a tail resume and a miss there mints a duplicate entry for a digest already held. A v2 sidecar is refused and rewritten rather than adopted, as v1 was: adopting one would leave lookup with neither a map nor a directory and answer null for held artifacts. Pinned by a test that downgrades a v3 blob and re-checksums it. The tests pin what equality cannot: that the map is EMPTY after adoption and every digest still resolves, that a digest which is not held returns null rather than a neighbour -- the failure a binary search has and a map does not -- and that digests written out of sort order still resolve, so a directory that merely mirrored insertion order would fail. Closes #145 closed check-in: 0d16bc966d user: dmestas tags: index-v3-145 | |
| 17:53 |
crc32: fold with ARMv8's crc32 instruction where the target has it
Checksumming was half the cost of opening a repository: 18.8ms of a 37ms `ls` on a 302MB store, across the two sidecars (26.3MB derived-state image at 13.5ms, 9.7MB log index at 5.3ms). Both measured by doubling the work and taking the delta, so the number is the checksum and not its neighbours. ARMv8's crc32b/crc32x implement CRC-32/ISO-HDLC -- the same generator this file is already defined in terms of, in the same reflected convention -- so this is an instruction-count change and never a value change, which is what the module doc says every edit here must be. tcl opens in 24.5ms against 38.5ms, output byte-identical. aarch64 only, deliberately. x86_64's SSE4.2 crc32 instruction is CRC-32C/Castagnoli, a DIFFERENT generator; taking it because the mnemonic matches would orphan every store already written. x86_64 keeps the table path, which stays load-bearing rather than becoming a fallback nobody runs. The new test recomputes CRC-32 as a plain bit-at-a-time recurrence, independent of update and its tables, and holds both paths to it at every length 0..1024 -- all eight residues mod 8, including the lengths below the fold width. The two paths differ only in where the fold stops and the tail begins, so a divergence would appear at particular residues and nowhere else, which is the shape a spot check misses. Chunked updates are pinned too: the store checksums payloads in 4 KiB pieces, so a path correct only for single-shot input would corrupt exactly the largest artifacts. Refs #143 closed check-in: b82aacc805 user: dmestas tags: crc-hardware-143 | |
| 16:50 | merge index-borrow-143: adopted index entries slice the blob (#143) check-in: 3c3f936632 user: dmestas tags: trunk | |