Parents and children of check-in [776eae66e9]
|
2026-10-24
| ||
| 02:27 |
fix(sync): withhold pending pushed artifacts; judge them when filled (#471)
Third review of the seedcheck design: - Withhold everything recorded, not just refusals. Every serving path (gimme/file, inventory and private igots, both clone paths, the client's file cards) now withholds any rid with a seedcheck row, pending or refused. A seed whose base a local check-in filled was served unjudged before. - Judge promptly. seedcheck.JudgeReadable judges, in one transaction, every pending row that has become readable, of any type: a seed is refused, anything else (files and non-artifacts too) loses its row, so withheld rows do not grow without bound. It runs in the transaction of each path that fills a basis: the dephantomize cascade, the sweep and rebuild (prepareCrosslinkTables), manifest.Checkin and tag.AddTag. Rows are walked in rid order through one cache, so a pending chain is expanded once. A row whose bytes fail their hash stays pending and withheld. - Forget on purge. shun.Purge drops seedcheck rows whose blob is gone (seedcheck.ForgetDeleted), since rids are reused. It is the only place blob rows are deleted. - One existence lookup per transaction. linkArtifact caches whether the table exists in the batch schedule, which is reset on commit and rollback. - Inventories read the withheld set fresh. The per-request cache serves gimme and clone only. Rows a request adds are its own unreadable deltas, which nothing can send, so the store-time and pre-inventory cache resets were unobservable and are gone. - The unsent delete on refusal is removed: only local writes fill unsent, so a pushed artifact is never in it. Tests: - The check-in-fills-base case no longer sweeps explicitly. It shows the seed refused in the check-in's transaction and served to neither a clone nor a pulling peer. - A pending seed is not answered on gimme or announced. - A pending plain file is served once its base arrives. - A purged seed's rid, reused by an innocent blob, is not withheld. - A private inventory withholds a privately pushed seed. - A clone does not send a refused seed as a delta source. - Disabling any serving filter fails a test. A clone of any store holding a dangling delta stalls at clone sequence 1. That is pre-existing and unrelated to seeds, so the pending-seed test checks pull and gimme instead. check-in: c72c1245b8 user: [email protected] tags: main, trunk | |
| 01:44 |
fix(sync): judge pending pushed seeds at the crosslink step (#471)
Re-review found two faults in the shun-based design: - Only the push path judged a dangling seed. A base filled by a pull, a local check-in or a later sweep still crosslinked the seed. - Shunning the seed was wrong. The shun table syncs through the shun config group. A horizon clone that pushed its own current seed back as a dangling delta could get the origin to shun that hash, and every clone pulling the group would then drop its seed. New design, in internal/seedcheck: - A store-local seedcheck(rid, refused) table, created on first use and carried by no configuration group. - The push handler records a delta it cannot expand as pending. Only pushed artifacts are recorded: a seed-shaped artifact that arrives by any other route is ordinary (B.2). - One choke point: manifest.linkArtifact. AfterDephantomize, the sweep, rebuild, the receive linker and required-content linking all reach it. There a pending control artifact is judged: a seed becomes refused, is not linked, and leaves unclustered and unsent. Anything else loses its row and links as usual. A pending non-control row is left, inert. - A seed judged within the request that carried it still refuses that request atomically (ownSeedReadable). It expands through the pass's basis cache, so a dangling chain is expanded once along its length. - Serving honours refused: gimme/file, inventory and private igots, both clone paths, and the client's file cards when it pushes on. judgeGrounded, the shun use and the shun filters added to the sweep, rebuild and cascade are removed. Tests: - The pull-fills-base probe and a check-in-fills-base case. - A refused seed not cloned, and not pushed on even when a peer asks for it by name. - The seedcheck table is in no config group. - B.4.4 also asserts the real minted seed is refused while the shun table stays empty. - Disabling the choke point, or any of the serving filters, fails a test. Clone receive is not covered: a fresh clone has no pending rows, so that path cannot meet one. check-in: 776eae66e9 user: [email protected] tags: main, trunk | |
| 01:18 |
fix(sync): judge a seed when it first becomes readable; shun late seeds (#471)
Review found that a seed pushed as a delta against a missing base, then completed by a later request carrying the base, was crosslinked with no check. Per the amended erratum (spec F), the test now runs on the artifact when it first becomes readable, whatever request carried it: - Full text and a delta on a readable basis are judged as they are stored. A delta left dangling is judged when a phantom grounding it is filled: storeFile runs storeReceived, then judgeGrounded over the filled rid's dependent deltas, before the after-dephantomize cascade. - A seed the current request carried refuses that request atomically with the single error card, as before. - A seed an earlier request left dangling is recorded in the shun table. The request that supplied its base is innocent and is stored and crosslinked normally, with no error. Refusing it would let anyone block a future push by planting a dangling seed delta against a hash the origin will later receive. - A dependent whose bytes do not hash to its name is skipped. Shunning the name would let a forged delta bar the real artifact. Why the shun table: every serving path already withholds shunned names (gimme/file, igot inventories, clone), receipt drops them, and shun.Purge can remove the content. What it does not cover was crosslinking, so candidateQuerySQL (sweep and rebuild) and the dephantomize cascade's delta-child walk now skip shunned blobs. That matches fossil, where shunned content never reaches crosslink. Two consequences: the entry propagates to peers that pull the shun config group, which only shuns an attack artifact, and an admin can lift it with shun.Remove. Also from the review: - handleFile's post-store bookkeeping moves into recordStored. - isSeed and the new helpers assert their inputs. - The end-of-request pushedSeed pass is gone; each delta is judged once when it becomes readable. Tests: the two-request case is now B.4.4, a unit test and conformance case horizon.5.2.3.seed-base-later-request. The later push succeeds and the seed is shunned, never crosslinked (rebuild included), sent or announced. A new unit test judges every dependent, ordinary before seed, and checks a forged dependent is not shunned. Mutations in the judging loop were checked and are killed. check-in: 3b76d8e5f5 user: [email protected] tags: main, trunk | |