This Day In History For 2027-07-30
1 Year Ago (more context)
|
2026-07-30
| ||
| 21:47 |
release_assert: a guard against assert used as input validation
std.debug.assert is right for an internal invariant and wrong for a rule about data the function did not produce, and it is wrong in two different ways. Under the ReleaseSafe default it survives and PANICS: on the write path that is the trade we want, a crash over a durable bad frame, but on a peer-reachable path it is a remote denial of service. Under an explicitly chosen --release=fast it compiles to nothing and the input flows through unvalidated. One fix, a checked error, closes both. Flags an assert only when all of: the fn is pub, returns an error union so a rejection was already available, the assert names one of its own parameters, and no earlier return in the body already rejects on that parameter. That last rule is what makes it signal -- delta.applyCopy returns MalformedDelta and then asserts to restate it, which is the CORRECT shape and must not be flagged. The baseline is keyed on file, function and COUNT, never on line numbers: line numbers churn on every edit above them, while a count still catches a second bad assert added to a function that already has one. It fails in both directions, so a fixed finding must be removed from the list and the list can only shrink. Existing findings are recorded, not fixed. Triage belongs in issue 19, where the ReleaseSafe default already reduces the write-path subset to crash-instead-of-corrupt and leaves the sync-side ones as the live exposure. check-in: 949e2b5542 user: dmestas tags: trunk | |
| 21:20 |
build: --release means ReleaseSafe, and pin it
src/ carries 1086 std.debug.assert calls. They are how a detected inconsistency becomes a crash instead of a durable write -- TigerStyle's 'downgrade catastrophic correctness bugs into liveness bugs' -- and that trade exists only while the assertion does. ReleaseFast and ReleaseSmall compile every one to nothing, so a binary built that way converts each guarded corruption into an unguarded one, in a store where a bad frame is permanent and a crash is not. Measured, not assumed. Read-side verbs are indistinguishable between the two modes: status, log and stat all sit inside noise on a 5 MB store. zit pack is not: 599ms fast against 15.2s safe, about 25x, because the delta encoder's best-of-N search is a tight per-byte loop where a bounds check is a large share of the work. That argues for setRuntimeSafety(false) on that measured loop -- local, reviewable, over code the fuzz and DST lanes already cover -- rather than stripping assertions from every other path in the binary. Not taken here. Note the option surface changes: stating a preference replaces the -Doptimize enum with a -Drelease bool, so the deliberate escape hatch is the top-level --release=fast. -Doptimize=ReleaseFast no longer parses. ci_test pins the preference, and says that if the project ever decides to ship unchecked, the pin should be deleted in that same commit so the decision is visible in history rather than arriving as drift. check-in: 4379187a10 user: dmestas tags: trunk | |
| 20:21 |
logstore: bound the priority window structurally, not with an assert
gatherCandidates capped the entries ADDED from the caller's priority slice, not the elements read: a duplicate hits ridListed and continues without advancing n, so the fill check never fires and the walk runs the caller's whole array with a linear dedup scan at each step. The only thing holding that back was an assert that the slice fits the window -- and std.debug.assert compiles to nothing in ReleaseFast and ReleaseSmall, so there was no guard in exactly the build where an unbounded scan would matter. Slicing the prefix holds in every mode, and turns an over-length slice from a precondition violation into defined behavior: accepted, tail ignored. Both callers today are inside the window either way. Two tests, both mutation-checked against the old shape. What is NOT pinned is the duplicate-heavy case: dedup leaves the surplus slots to the chronological walk, so the candidate set is identical either way and only the count of elements read differs. That is a cost property, not a behavioral one, and no test through this API can catch it -- the header comment says so rather than letting the pair read as covering it. check-in: bfec65b1a1 user: dmestas tags: trunk | |
| 18:47 |
merge shun-implementation: shunning and space reclamation
A repository-local shun table (sync SS3.3), the zit shun verb over it, and the four suppression directions the drafts require: never requested, never announced or served, an arriving copy discarded, and the bytes reclaimed on demand through the existing atomic rewrite. Built as eight slices, each through design review, TDD, spec audit, three review lenses, mutation proof and a full gate. Reading the eight as one change afterwards found what none of them could alone: a peer-reachable assert that crashed the client on a malformed DELTASRC, and a pre-rename guard that re-stat'ed a stale handle so two concurrent rewrites could un-shun each other. Two defects predating the feature are fixed on the way. FileStorage.sizeImpl answers from a cached watermark, so pack's tail-revalidate compared a key against the number it was derived from and could never abort -- the guard had never worked. And a malformed file card could trap the client through PendingDeltaQueue. Format version stays 1: an older binary meets an unknown frame kind and rejects cleanly, which is what it should do with a tombstone it cannot read. No interchange surface changed -- smoke covers ten fossil 2.28 scenarios including a fossil clone of a defer-shunned store and a real fossil push of a shunned artifact. Known and recorded, not blocking: a running zit serve keeps serving the old inode until restarted after a --force reclaim swaps generations. check-in: e57bcceadf user: dmestas tags: trunk | |
| 18:32 |
shun: whole-branch verification fixes
Read as one change rather than eight slices, which surfaced defects no per-slice review could see. A peer could crash us. ShunSet.contains asserted its key domain while its own vtable contract promised an out-of-domain name answers false and never traps -- and a 'file HASH zz SIZE' card reached it. It is a checked rejection now, and receiveDelta refuses a DELTASRC that is not a complete artifact name before one can mint a phantom, which also closes a pre-existing PendingDeltaQueue trap on the same card. Two rewrites could un-shun each other. rewriteAndSwap's pre-rename guard re-stat'ed the caller's handle, which after a competing rename points at the old inode -- so the guard passed and the loser silently reverted the winner. It reopens .zit by name under the temp lock now, as shunAppendRows already did. The client snapshot lacked the masked-basis guard the serve path has, so a kept delta rooted on a shunned base could be handed out in delta form. shun_help denied two slices that landed after it was written: it still told operators a shunned artifact is only suppressed in one direction and can be re-announced back into the store. Also: storestat's shun rows and bytes are wired into zit stat rather than left unused, so the printed breakdown still sums to the file size; the clone-send half is relabelled an erratum candidate against SS8.1/SS8.2 instead of claiming SS9.5, which is scoped to non-clone; and three stale doc bullets now describe the code that exists. closed check-in: d9578c6c6e user: dmestas tags: shun-implementation | |
| 17:20 |
shun S7: the shun+reclaim crash sweep
A DST sweep that shuns artifacts (forcing a delta base, a manifest and a mid-chain artifact across seeds), reclaims through rewriteAndSwap, and cuts the run at sampled write/sync/rename boundaries. The oracle classifies the recovered store as the PRE image or the POST image and fails on anything else. It earns its place: deleting packLog's pre-rename fsync is caught by this sweep and by nothing else in the suite. The concurrency-guard boundary is a second, DST-shaped pin on S3's fresh-stat fix rather than first coverage, and the module header says so. Two oracle clauses are graded honestly as tripwires, not coverage. Surviving content is re-hashed by packLog before its own commit, so a defect that would corrupt it aborts the rewrite instead and the sweep just sees the PRE image; no mutant can make that clause fire. The PRE half of the third clause is subsumed by the byte-equality check. Both are kept cheap and labelled. rewriteAndSwap and RewriteError are now pub: the property under test is a property of that sequence and reaching it needs the seam. faulty_dir gains fileSetLength and fileClose wrappers, deliberately ungated so no sibling sweep's cut numbering shifts. The default dst gate grows about 11s (~9%) for 12 sampled cuts per seed. check-in: cb22503707 user: dmestas tags: shun-implementation | |
| 14:53 |
ignore the crash-leftover working files: dst workspace and pack temp
.zit-dst-checkout-ws/ is the process directory the crash-recovery sweeps make their throwaway workspaces under. It is created at run start and removed by a defer at run end, so it is normally invisible -- but a killed or crashed sweep leaves the whole tree, .zit stores and all. .zit-pack-tmp is the fixed-name temp pack and shun --force write the new generation into before the atomic rename; a run that dies between create and rename leaves one beside .zit. Both are dotfiles, so a bare 'fossil extras' skips them and the tree reads clean while they sit there -- the same blind spot the .agent-config glob closed. Pinned by packaging_test. check-in: 9b95e3b9e5 user: dmestas tags: shun-implementation | |
| 13:15 |
shun S6: receive-side admission, asymmetric as the spec has it
SS7.7 L1434-1452, as one combined predicate: a target is shunned for admission when its complete name is in the shun table OR shun-sha1 applies to its SHA1 name. TargetAdmission.admits carries the whole rule, including the two card rules -- a clone file card BYPASSES the check and is stored; everything else consults it. That asymmetry is deliberate and is the applied text. zit's own erratum argued the discard should apply in clone too and four reviews rejected it against fossil's xfer.c. A test pins the bypass so a later reader does not 'fix' it back into the shape that was refused. For cfile the SS7.5 private discard runs first and the raw artifact-card count still increments -- both satisfied by placing the check below code that already existed, not by adding any. Admission is target-only. A shunned DELTASRC is not independently rejected: a held basis still serves, and an unavailable one falls to ordinary dangling-delta handling whose phantom SS3.3 already excludes from gimme. The planned e2e did not pin anything and was rewritten. A real zit push passes with the whole check removed, because transfer is receiver-driven and SS3.3 already stops this server begging for a shunned name. The shipped test POSTs an UNSOLICITED push+file container at a live serve -- the only shape that reaches this code -- and it goes red without it. Smoke S10 is real interop coverage but cannot pin this slice for the same reason, and its header says so rather than letting a green be misread. check-in: 620ed6a9c3 user: dmestas tags: shun-implementation | |
| 10:34 |
shun S4: stop advertising what the shun table names
LogIndexStore keeps a shunned holding in its enumeration and MASKS it rather than removing it, so a shun row appended between two rounds of one clone cannot renumber the survivors under the pager's cursor. held() answers false, the name leaves by_name, and SS9.5's 'igot for every eligible holding after the SS3.3 filters' is where that mask lives. The plan overstated what was missing. S1's log-level mask already nulled content and wire, so a clone silently skipped a shunned artifact before this slice. What was actually broken is narrower and worse: because by_name still resolved the name, emitCloneReply's basis proof fired in a safety build and the SERVE PROCESS PANICKED -- a fossil clone got 'server did not reply', not an absent artifact. Reproduced against fossil 2.28 and pinned as smoke S9. That also means the planned e2e ('a clone does not receive it') was never mutation-sensitive: it passes with this slice fully removed. The observable that pins the slice is the ABSENCE of an igot card naming a shunned artifact in a non-clone round's inventory walk, which now has its own test and a second round in the raw-HTTP e2e case. Smoke uses a defer-shunned fixture, not the planned packed one: pack rebuilds the delta chains and so removes the exact hazard the check exists to catch. logstore.zig carries a comment-only change: LogStore.shunned's doc claimed two questions were open that S4 and S5 have since closed. check-in: 230c2bbebe user: dmestas tags: shun-implementation | |
| 03:50 |
shun S3: the zit shun verb
Refuses by default and names what would be destroyed; --force proceeds; never prompts. Reclaims by default -- one rewrite for N hashes, which is why the usage takes HASH... --defer records the tombstone without rewriting, and is also gated by --force: the row is irreversible and S1's mask hides the content immediately, so it is not the safe half of the verb. The check-in this checkout is on cannot be shunned at all, with no override. runShun does NOT reuse runPack's plumbing verbatim as planned -- it diverges on the filter (the source's deferred rows UNION the newly named hashes), on writing tombstones into the new generation, and on its renderer. The shared body is extracted as rewriteAndSwap instead. Fixes a dead concurrency guard in that shared body, which predates this slice. FileStorage.sizeImpl returns a cached watermark rather than stat'ing, so validateTail was comparing a key against the number it was derived from and could never abort. pack carried the same dead guard. rewriteAndSwap now stats fresh at the check point; without that, this slice's claim that an aborted --force leaves no tombstone was unprovable. Consequence lines carry no counts or filenames: 'N descendant check-ins' and a blob's path both need a reverse F-card walk over every manifest, which is too much for the cheap refusal path. Deferred, not faked. No sub-manifest row: that type and its draft are not on this branch. shunConsequence is a total switch, so a ninth type will not compile without its diagnostic. check-in: be67814135 user: dmestas tags: shun-implementation | |
| 02:45 | feat(submanifest): add parser and sparse editor check-in: 8ca8c86174 user: dmestas tags: sub-manifest | |
| 00:39 |
shun S2: pack reclaims what the shun table names
packLog takes a borrowed Filter; a dropped rid is not copied, PackStats.dropped counts it, and packVerifyShunned refuses to swap in a store still holding a digest the filter was told to remove. keep_all is the identity, so every other caller behaves as before. The check runs BEFORE reconstruction and before the SS7.7 verify. A shunned artifact may be unlawful or simply broken, and verifying it would abort the very operation meant to remove it. Two tests pin the ordering: one frame whose content no longer hashes to its name, one that will not decode at all. The second closes a hole the first left in deriveCandidateHints. putArtifactPacked's floor_span became optional. A delta whose base the filter dropped has no keepable frame, and 'I have no floor' is a different claim from 'my floor spans N bytes'; an in-band sentinel could not tell that from a miscomputed span. pack_help no longer overclaims: damaged CONTENT does not block the reclaim, a damaged CHECKSUM still does, because the open-time pre-flight covers the whole store before any reclaim runs. Merges S5's e2e block alongside this slice's. Fossil's suggested resolution would have dropped this one. check-in: 341009e699 user: dmestas tags: shun-implementation | |
| 00:25 |
shun S5: never gimme a name in the shun table
sync SS3.3 L527-530 excludes shun-table names from the ordinary phantom scan. The predicate already existed and its call site hardcoded false; S5 supplies the durable state through a borrowed ShunSet whose .none identity leaves every existing construction site unchanged. The literal reading is kept: an igot for a shunned hash still CREATES the phantom (SS3.3 L511-514) -- it is simply never requested, so the phantom is inert and permanent. Three consequences the reviews forced out: A shunned phantom no longer counts as unresolved. It was told about and refused, not asked for and withheld; counting it made zit pull fail on every future pull, since the peer re-announces each round. The server consults the set too. SS3.3 states the filter on the phantom scan without qualifying a side, and a server under push-receive is a receiver, so an inbound igot could otherwise walk a shunned artifact straight back in. parseIgotCard now rejects a name that is not 40 or 64 hex, as parseGimmeCard already did (artifact-format SS3.2 L349-350). Widening the shun predicate to answer false for an out-of-domain name would have been the fail-open fix. check-in: 1c41f96e86 user: dmestas tags: shun-implementation | |