Artifact 3976c4114ec67a0b81e49bc0b8a7178f109a02d9cd43443a3c674854253a0c81:
- File
internal/manifest/schedule.go
— part of check-in
[c72c1245b8]
at
2026-10-24 02:27:29
on branch main
— fix(sync): withhold pending pushed artifacts; judge them when filled (#471)
Third review of the seedcheck design:
- Withhold everything recorded, not just refusals. Every serving path (gimme/file, inventory and private igots, both clone paths, the client's file cards) now withholds any rid with a seedcheck row, pending or refused. A seed whose base a local check-in filled was served unjudged before. - Judge promptly. seedcheck.JudgeReadable judges, in one transaction, every pending row that has become readable, of any type: a seed is refused, anything else (files and non-artifacts too) loses its row, so withheld rows do not grow without bound. It runs in the transaction of each path that fills a basis: the dephantomize cascade, the sweep and rebuild (prepareCrosslinkTables), manifest.Checkin and tag.AddTag. Rows are walked in rid order through one cache, so a pending chain is expanded once. A row whose bytes fail their hash stays pending and withheld. - Forget on purge. shun.Purge drops seedcheck rows whose blob is gone (seedcheck.ForgetDeleted), since rids are reused. It is the only place blob rows are deleted. - One existence lookup per transaction. linkArtifact caches whether the table exists in the batch schedule, which is reset on commit and rollback. - Inventories read the withheld set fresh. The per-request cache serves gimme and clone only. Rows a request adds are its own unreadable deltas, which nothing can send, so the store-time and pre-inventory cache resets were unobservable and are gone. - The unsent delete on refusal is removed: only local writes fill unsent, so a pushed artifact is never in it.
Tests: - The check-in-fills-base case no longer sweeps explicitly. It shows the seed refused in the check-in's transaction and served to neither a clone nor a pulling peer. - A pending seed is not answered on gimme or announced. - A pending plain file is served once its base arrives. - A purged seed's rid, reused by an innocent blob, is not withheld. - A private inventory withholds a privately pushed seed. - A clone does not send a refused seed as a delta source. - Disabling any serving filter fails a test.
A clone of any store holding a dangling delta stalls at clone sequence 1. That is pre-existing and unrelated to seeds, so the pending-seed test checks pull and gimme instead. (user: [email protected], size: 4404) [annotate] [blame] [check-ins using]
package manifest import ( "slices" "github.com/danmestas/go-libfossil/db" libfossil "github.com/danmestas/go-libfossil/internal/fsltype" "github.com/danmestas/go-libfossil/internal/leaf" "github.com/danmestas/go-libfossil/internal/repo" ) // batchSchedule is the work a crosslink batch defers to its repair gate: the // leaf rechecks its branch writes schedule (leaf.Pending, // draft-fossil-repo-state-01 Section 7.3), the primary parents whose tags // must be propagated again because the batch gave each of them a child, and // whether it linked anything since its last gate, which owes the reparent pass // even when it scheduled nothing else. All three describe writes in the // batch's transactions, so they are marked, rolled back and reset together. // // It is not safe for concurrent use; one batch owns one. type batchSchedule struct { leaves *leaf.Pending parents parentSet owed bool // seedTable caches, for one transaction, whether the seedcheck table // exists (0 not yet asked, 1 absent, 2 present), so linking does not // look it up per artifact. seedTable int8 } // linkTx runs link in one transaction on r and, when link linked anything, // runs the whole repair gate (repairOnTx) before that transaction commits: // the reparent pass, tag propagation from the parents sched holds, and the // leaf rechecks sched holds. Fossil's manifest_crosslink_end does the same // inside the transaction that linked the check-ins (src/manifest.c; its // leaf_do_pending_checks runs before db_end_transaction). A link and the // repair it owes therefore commit or roll back together: no committed // check-in waits on a later gate for its parent's tags (issue #432) or its // leaf row (draft-fossil-repo-state-01 Section 7.3). On commit sched is // spent; on rollback everything this transaction scheduled is dropped. func linkTx(r *repo.Repo, sched *batchSchedule, link func(tx *db.Tx) error) error { if r == nil { panic("manifest.linkTx: r must not be nil") } if sched == nil { panic("manifest.linkTx: sched must not be nil") } mark := sched.mark() err := r.WithTx(func(tx *db.Tx) error { if err := link(tx); err != nil { return err } if !sched.owed { return nil // linked nothing: no gate is owed } return repairOnTx(tx, sched) }) if err != nil { sched.rollback(mark) return err } sched.reset() return nil } // scheduleMark is a point a batchSchedule can be rolled back to. type scheduleMark struct { leaves leaf.Mark parents int owed bool } func newBatchSchedule() *batchSchedule { return &batchSchedule{ leaves: leaf.NewPending(), parents: parentSet{seen: make(map[libfossil.FslID]struct{})}, } } // mark records the schedule's current extent, for a caller whose transaction // or savepoint may roll back. func (s *batchSchedule) mark() scheduleMark { return scheduleMark{leaves: s.leaves.Mark(), parents: len(s.parents.order), owed: s.owed} } // rollback forgets everything scheduled since m. func (s *batchSchedule) rollback(m scheduleMark) { s.leaves.Rollback(m.leaves) s.parents.truncate(m.parents) s.owed = m.owed s.seedTable = 0 } // reset empties the schedule once the gate that ran it has committed. func (s *batchSchedule) reset() { s.leaves.Reset() s.parents.truncate(0) s.owed = false s.seedTable = 0 } // parentSet holds the primary parents a batch wrote plink rows for, in the // order it first recorded them, so a rollback drops exactly what it added. // They are the parents repairTagPropagation starts from (issue #432). type parentSet struct { seen map[libfossil.FslID]struct{} order []libfossil.FslID } // add records pid, a primary parent whose plink row the batch just wrote. func (p *parentSet) add(pid libfossil.FslID) { if pid <= 0 { panic("manifest.parentSet.add: pid must be positive") } if _, ok := p.seen[pid]; ok { return } p.seen[pid] = struct{}{} p.order = append(p.order, pid) } func (p *parentSet) truncate(n int) { if n < 0 || n > len(p.order) { panic("manifest.parentSet.truncate: mark is past the set's end") } for _, pid := range p.order[n:] { delete(p.seen, pid) } p.order = p.order[:n] } // sorted returns the parents in ascending rid order, the order the // whole-repository pass visited them in, so a gate's writes are reproducible. func (p *parentSet) sorted() []libfossil.FslID { out := slices.Clone(p.order) slices.Sort(out) return out }