Skip to content
zit Today In History
Login

Today In History

This Day In History For 2027-08-07

1 Year Ago (more context)

2026-08-07
21:50
update: pin the fail-closed refusal of a foreign check-in

Regression tests for issue 42. A blob whose bytes are a valid check-in manifest IS a check-in -- structure types artifacts, and repo-state 4.4 exempts only what a parser declines to classify -- so a repository that tracks such a file holds check-ins of another repository. That derivation is required; what must not regress is the refusal.

Asserts materialization fails with FileContentMissing, names the file, and leaves nothing written -- the fail-closed property stated as a test so it cannot be quietly traded later. Also pins that the diagnostic explains the class and never alleges corruption, since the store is intact and verify says so.

Records the T-card grammar that makes such a manifest resolvable: 'T *branch * trunk' self-targets, while 'T *branch trunk' puts a branch name in the UUID field and the manifest is rejected entirely. (#42) check-in: 0369e62f19 user: dmestas tags: sub-manifest

21:42
open/update: refuse a tip whose content this repository never held

The format types artifacts by structure, and repo-state 4.4 exempts only what a parser declines to classify -- so a stored blob whose bytes are a valid check-in manifest IS a check-in, joins the DAG, and may carry its own branch tags. A repository tracking such a file (a Fossil-to-Git mirror's 'manifest' is the common source) therefore holds check-ins of another repository, whose blobs it never had. That derivation is conformant; Fossil does the same.

What was not acceptable was spending minutes materializing before finding out, then reporting it as a missing file with no way to see why. open now presence-checks the resolved tip's file set before invoking the engine and refuses naming the class; update prints the same explanation.

Advisory only: a files() fault falls through, and stagePlan remains the sole authority, so this cannot let anything through -- it can only refuse earlier. Verified against a foreign self-tagged manifest that genuinely hijacks trunk: exit 1, working tree untouched. (#42) check-in: a81e9a8745 user: dmestas tags: sub-manifest

21:10
update: cover the legacy trailing-separator plan path

parseForImport normalizes a fossil-origin F card into a fresh buffer and the manifest borrows that copy, so every F-card hash the plan reads points into it rather than into the stored bytes. Nothing exercised it: every synthetic corpus here is git-native and never carries the separator, so normalized is null throughout the suite.

The path is correct -- the test passes -- but it was covered only by repositories nobody runs the tests against. (#42) check-in: 15db533ea7 user: dmestas tags: sub-manifest

20:57
open: name the check-in it resolved when materialization fails

open picks its own target -- the tip of the configured main branch -- rather than being handed one, so a failure that omits the check-in cannot distinguish materializing the expected one going wrong from resolving a different one than the operator believes is the tip. There was no other way to see which. (#42) check-in: da081a3eca user: dmestas tags: sub-manifest

18:59
update: copy the staging fault's path instead of borrowing freed memory

StageFault held a slice into PlanEntry.path, documented as borrowing a plan that outlives the call. It does not: applyCheckin frees the plan from its own defer on the way out, so the caller rendered freed heap -- ten bytes of garbage where a filename belonged, in the very diagnostic added to make this failure legible.

Path is copied now, as the artifact name already was. Adds the end-to-end test that would have caught it: the earlier test hand-built a fault and so never exercised the lifetime that was wrong. check-in: 41dc7067d8 user: dmestas tags: sub-manifest

18:20
update: name the file staging cannot materialize

Three separate sites printed the identical 'cannot read repository: repository is corrupt' -- deriving the view, reopening the log, and applyCheckin. So a materialization failure was indistinguishable from a failure to read the repository, which is a different claim about a different subsystem.

stagePlan's three refusals now return FileContentMissing carrying the file path and the artifact name: a file the check-in references has no artifact here, or has none stored, or the stored bytes do not match the name. verify cannot surface any of these -- it checks that stored artifacts are valid, never that referenced ones exist.

Being in the closed ApplyError set, the compiler required all four verb mappers to handle it. (#42) check-in: 0b7b4af7fe user: dmestas tags: sub-manifest

17:49
repoview: name the artifact a failed derive rejected

'repository is corrupt' says a repository cannot be read, not what in it is unreadable -- and the walk that discovers it visits every artifact in the store, minutes on a large one. Whoever holds the broken repository was left bisecting it or attaching a debugger.

The derive now records which artifact it rejected and why; openRepo reports it, so every verb's open gets the name. Plain data, no allocation, so it survives the errdefer that tears down a half-built view. Repository.open clears it per attempt because callers declare a Repository undefined.

Also splits open's two identical diagnostics: deriving the view and reopening the log for materialization both printed the same line, which made a failure in one indistinguishable from the other. (#42) check-in: 37f746e59f user: dmestas tags: sub-manifest

17:17
pack cache: derive the entry cap from the byte budget

A real 53,773-commit import evicted 99,097 times, every one forced by the 4096-entry cap and not one by the 64 MiB byte cap, with the resident set at 23,551 KiB. The cache was held to a third of its own stated size by a second, unrelated number, and the 51 percent hit rate that produced looked like a working-set problem rather than an arithmetic accident.

The entry cap now divides the byte budget by a planning entry size, so the two cannot disagree about which one is bounding: bytes bind for normal data, entries only for many tiny items, which is what an entry cap is for. (#40) check-in: ac89bad5fd user: dmestas tags: sub-manifest

15:29
import-git: name the main line trunk, whatever git called it

A git default branch is master or main; this format's is trunk, and so is every place that must name the main line without being told -- a fresh checkout, zit open resolving a tip, a clone initialising its own state. A repository whose main line was called master satisfied none of them: it cloned and verified clean, then could not be opened, because the branch the destination looked for held no commits.

labelBranches now takes the name to MATCH on and the name to APPLY separately: the head is still found by its source name, the segment is labelled trunk. Side branches keep their source names, and the git-side refs mapping still records sym-heads/master, so nothing about round-trip identity moves.

If a side branch already publishes trunk, the source name is kept rather than colliding two segments under one label. (#42) check-in: 5caf866d86 user: dmestas tags: sub-manifest

13:03
storage: a block cache for FileStorage reads, opt-in and not yet wired

Profiling a real pkgsrc-scale import put readv at over half of all CPU samples. A block cache in front of pread is the shape that addresses it: the store is append-only, so a block is retained only when the read that filled it came back FULL -- which proves those bytes already existed, so appends cannot alter them and need no invalidation. Only truncate does.

Deliberately not attached to any caller. Wired into import-git it reached a 99 percent hit rate over 392,421 reads and did NOT make a small corpus faster: at that size the page cache already served those reads, so the syscalls were cheap and the lookup was pure overhead. The prior from the real profile is strong but unproven at scale, and shipping it on 'should help' is how the reverted top-level cache in issue 32 happened.

useReadCache is the one-line opt-in for profiling it. (#39) check-in: f3becbedd1 user: dmestas tags: sub-manifest

11:34
pack cache: name WHICH budget forced each eviction

A real 53,773-commit run reported 99,097 evictions against 69,485 lookups -- evicting more than it was asked about. But two caps can force that, max_bytes and max_entries, and they bind under different data, so the count alone said only 'raise something'.

Evictions are now split by cause and the resident set is reported, so a run names the budget to raise instead of requiring a blind sweep. (#40) check-in: 88ae8ba619 user: dmestas tags: sub-manifest

03:25
import-git: report the pack plaintext cache the way the memo is reported

PackStats carries the cache's hits, misses, evictions and oversized skips, and import-git prints them. Same reason SubManifestMemo got this in #35: a fixed budget that is wrong for a corpus is invisible from outside the process, and a hit count without its denominator cannot show it. Nothing here gates a decision. (#40) check-in: b6cd09fc11 user: dmestas tags: sub-manifest

03:24
logstore: one read for a frame's payload and its CRC trailer

The trailer is the four bytes immediately after the payload and was read by its own pread. FileStorage is unbuffered, so that was a second kernel transition for adjacent bytes, paid per frame -- and a deltified artifact walks its whole chain frame by frame.

Also gives PlaintextCache the counters it was missing. It tracked hits with no denominator, so a hit COUNT could not be read as a hit RATE; misses, evicted and oversized make the budgets observable. (#39, #40) check-in: a4f298d3e1 user: dmestas tags: sub-manifest

02:37
managed: derive the entry and path-table bounds from the format

max_entries was 1 shifted by 20 while its own doc named the u32 count field that carries it, so insert refused a correct set at 1,048,576 paths -- four thousand times below what the format can express. Now the u32 range, with max_file_bytes computed from it rather than a flat 64 MiB, and read sized from stat like every other whole-file read here.

Two dependents moved in the same change, which is the point:

extras borrowed max_entries as a directory-sweep ceiling. Never the same quantity, and a tracked-set format change would have silently widened a filesystem runaway guard. It has its own now.

undo parseSlot allocated count entries BEFORE parsing them, guarded by count over max_files. That guard became vacuous the moment max_files reached the u32 range, leaving a 4-billion-entry claim reaching the allocator. Now bounded by what the remaining payload can back. (#38) check-in: 93adddd49d user: dmestas tags: sub-manifest