Timeline

A structural clone holds the manifests of the held set and everything they reach through B and S cards (draft-fossil-horizon-00 §5.1), so a delta manifest's baseline below C is held. Its parents were labelled by nobody: boundaryOf took the parents of K's check-ins only. The clone then classified such a parent as transit and requested it, and the origin, which withholds manifests below C, never sent it; the clone stalled with one phantom outstanding. zit's origin has the same gap (sparse.zig 758-783), and zit's author agreed the draft will define a boundary hash as an unheld parent of any manifest the clone holds. boundaryOf now labels the parents of K and of every check-in the reach holds.

The regression is the HZ-3.1-4 case for a delta manifest in K whose baseline lies below C: the clone completes, the baseline is held but not a leaf, and K's check-ins derive the dense repository's tags. check-in: f0354673e2 user: [email protected] tags: feat/horizon-cones, trunk

The origin's classification memo (draft-fossil-horizon-00 §6.2): the reference graph and each cone's classification kept per store prefix and extended by appended artifacts, recomputed in full only when an arrival was already cited. A pull after a one-artifact push on fossil-scm.org's repository answers in about 0.25 s instead of 2.7 to 4.5 s. check-in: 7dec3e4fb3 user: [email protected] tags: feat/horizon-cones, trunk

HZ-6.2-9 and HZ-6.2-10 were gaps. The new cases read, from the handler's report of what each request's classification read, that a pull over an unchanged store reads nothing, one after a commit reads what it appended, a blob after its manifest is classified alone, a parent after the child citing it forces a full read, and a cone never classified is classified without re-reading the store; and that every reply announces what a second handle with an empty memo announces. HZ-6.2-10 is partial: a horizon's first classification walks its held set's trees from the store, as zit's does.

The case list joins the horizon suite from the coverage table, leaving the horizon case files to the work in flight there. check-in: 6f8176bb93 user: [email protected] tags: feat/horizon-cones, trunk

A store crosslinked only locally has no non-artifact record until its first received artifact is crosslinked, so the first push to an origin made the memo read the whole store again. The record appearing is every version it names being named at once; those were read raw, and reading file content raw adds nothing, so the extension takes them as files and goes on. Only a version whose raw read added a record or an unreadable name still forces a full read. check-in: bdff158325 user: [email protected] tags: feat/horizon-cones, trunk

Measured on a dense 68k-artifact, 21k check-in repository, a horizon pull after a one-artifact push still spent most of its time scanning every blob's metadata to extend the memo, and a structural request extended it twice: once to place the horizon ahead of the request's transaction, unpublished, and again inside it.

An extension now carries the artifact names over and reads metadata only for the arrivals and the delta ancestors their reads expand. A structural request places its horizon in a read-only transaction of its own, a committed snapshot, and publishes that view, so the inventory finds the memo current.

TestCorpusHorizonPullAfterPush (LIBFOSSIL_CORPUS) times it: a pull after a one-artifact push goes from about 2.7s to 0.2s for a content horizon, and from about 4.4s to 0.3s for a structural one. check-in: 62ffc8ff92 user: [email protected] tags: feat/horizon-cones, trunk

Each request re-read the store to classify its cone, so a pull after a one-artifact push to a large origin cost a full read. The handler now takes each store's classification memo (one per pool DB, dropped with it) and classifies its inventory and its structural reach through a view of it. The view is published only once the request has committed and stored no artifact: the inventory classifies before received files are crosslinked, so such a view describes a store that never committed. A caller holding its own transaction (a bound DB) classifies afresh.

SparseClassified now reports what the classification read: the whole store on a full read, the arrivals on an extension, none when the memo covers the store.

A property test drives random commits in every manifest form, pushes in any order and announcements against an origin and checks, after each step, that the memo keeps exactly what a fresh classification keeps for path cones, content and structural horizons under both rules. check-in: 83f8e43ebd user: [email protected] tags: feat/horizon-cones, trunk

A go-libfossil store is not append-only: a phantom fills at its old rid, and the crosslinker can link an old artifact once what it waited on arrives. An extension by the appended and filled rids alone would miss that, so the memo counts the check-in and plink rows at or below its prefix and reads the store in full when they move other than by a fill, or when a filled phantom is the base of a stored delta (the delta was unreadable before the fill). A new check-in's mlink rows naming an old file version no longer force a full read: reading file content raw adds nothing to the facts, so only a version that did is a change. check-in: 423b5535d4 user: [email protected] tags: feat/horizon-cones, trunk

An origin re-reads its store for every horizon request: the reference facts behind the path classification and the trees of the whole held set. draft-fossil-horizon-00 §6.2 asks it to keep the classification keyed by cone digest and store prefix, extend it by the artifacts appended since, and recompute in full only when an arrival carries a name a held P, B or S card cited while it was absent.

Memo keeps the store's cone-independent references (the reference facts, the check-in graph and the dangling P/B/S names) at a prefix, and per cone digest and per horizon the classification over them. A view extends each by the appended and filled rids: the loader's decisions for those rids alone, classification of the facts' new tails, and the tree walk of the check-ins K gained. Anything other than an append or an uncited fill (a cited arrival, a deleted or phantomized row, a first tree) reads the store in full. Views are copy-on-write and reach the memo only when published, so a rolled-back request leaves nothing behind. check-in: e5bdac3d33 user: [email protected] tags: feat/horizon-cones, trunk

An origin computes K for every horizon request by reading every plink edge and held check-in from the store. draft-fossil-horizon-00 §6.2 asks it to read those references once into a graph keyed to the store prefix and extend that graph by the artifacts appended since. Graph is that graph: loaded once, extended copy-on-write by the rids that changed, and HeldSet over it equals HeldSet over the store. check-in: b1933e5f82 user: [email protected] tags: feat/horizon-cones, trunk

HZ-3.1-2: the leaves rule's cost is stated where it is chosen. HZ-3.4-2: a horizon reaches the wire only as a full hash and a rule name. check-in: c79a113e0c user: [email protected] tags: feat/horizon-cones, trunk

A client that wants to merge across live branches offline should pick the leaves rule knowingly (HZ-3.1-2). The rule holds every descendant of where each childless check-in's history meets C's, which on a repository with old unmerged branches is nearly the whole history. clone and cone store --leaves now warn once on stderr, the flag help says so, and HorizonLeaves documents the cost for library callers.

HZ-3.4-2 asks that a horizon never be a count or a date on the wire. A case records the request cards of a horizon clone and pull under both rules and asserts one pragma of a full hash and a rule name; the CLI's clone refuses a name, and its store cone resolves a branch name to the full hash first. check-in: d7a8b71b9b user: [email protected] tags: feat/horizon-cones, trunk

A sparse or horizon store of a real-sized repository rebuilds: deferred check-ins are linked after the sweep and its non-artifact record, so reference loading answers from the tables instead of reading every blob against the budget. check-in: ba1e308608 user: [email protected] tags: feat/horizon-cones, trunk

A sparse rebuild must load whole-store references once the sweep has filled the tables they are answered from, or it reads every blob raw and exceeds the reference budget. The loader is now a seam the test observes, and the corpus test checks every phantom classifies as before a rebuild. check-in: 1faf2d23c3 user: [email protected] tags: feat/horizon-cones, trunk

A sparse or horizon store's deferred check-in is linked partially, which loads whole-store references. Those are answered from mlink and the non-artifact record, but a rebuild empties mlink and records non-artifacts only at the end of the batch, so the first deferred check-in found nothing to answer from, read every blob raw and exceeded the 512 MiB expansion budget on a real-sized repository (Rebuild failed after about a minute).

linkBatch now sets deferred check-ins aside and links them partially once the sweep and the non-artifact record are done, when the references are answered from the tables as they are after a clone. A rebuild of the {src}-cone store now completes in about 50 s under 700 MiB; its derived tables are row-for-row identical to those of the previous code run with the budgets raised far enough to finish. check-in: 85a71d319a user: [email protected] tags: feat/horizon-cones, trunk

Rebuild of fossil-scm.org's repository in about 50 s instead of 3 to 6 minutes: delta-chain availability answered from the sweep's cache, and each manifest parsed once. Every derived table is identical before and after. check-in: 6339d6fb50 user: [email protected] tags: feat/horizon-cones, trunk

Rebuilding a clone of the Fossil SCM repository (67,894 artifacts, 21,008 check-ins) took about 3 minutes of which 104 s was parentTreeGap asking content.AvailableByUUID, an uncached delta-chain walk with one SQL statement per node, for every parent of every check-in. It now uses the sweep's availability cache like the rest of the guard.

The same sweep parsed each check-in three times (as a child, as its child's parent, as a delta baseline) and rebuilt its file maps. The sweep's content cache now keeps the parsed deck, file views and baseline rid of its few most recent check-ins (Cache.Derived), dropped with the cached bytes on Invalidate and Clear. Derived hits count as reuse in the cascade stats, so the shared-cache regression guard keeps measuring it.

The rebuilt event, mlink, plink, tagxref, leaf, filename, cherrypick, orphan, unclustered, unsent, forumpost, attachment, phantom, blob and delta tables are row-for-row identical to the previous code's. check-in: 48ff1f5084 user: [email protected] tags: feat/horizon-cones, trunk

The rebuild budget is a wall-time and memory target, and the test only reported time. Sampling HeapInuse beside the run makes a regression in either visible from one -v run. check-in: 7cb4d1aa0d user: [email protected] tags: feat/horizon-cones, trunk

Merges follow renames as fossil's merge does: its name pivot, find_filename_changes and file matching ported, so a file renamed on one side and edited on the other merges into the renamed file. Matches fossil 2.28 on 13 rename scenarios. Repo.Merge no longer brings back files the merge deleted. check-in: 4bccd3f803 user: [email protected] tags: feat/horizon-cones, trunk

A file renamed on one side and edited on the other was merged as a deletion plus an addition: the edit was kept under the old name and the renamed copy arrived beside it, so fossil and go-libfossil left different trees and recorded different changes for the same merge.

Port fossil's rename matching (merge.c merge_cmd): the name pivot nid (pivot_find(1) of M and V, rerun against P when they differ), the renames from nid to V, P and M (path.c find_filename_changes, one way, now path.FilenameChanges), and the fv table those fill (add_renames, the nid!=pid name fill-in, then the files of V, P and M) as merge.MatchFiles/FollowRenames. Checkout.Merge classifies each matched file as before and renames on disk and in vfile a file the merged-in version renamed and the checkout did not, after the other steps, moving swapped names together and setting aside a removed or deleted row that held the new name. Where fossil would rename over a kept tracked file or an untracked one, Merge refuses, as it already does for adds.

Repo.Merge matches files the same way and commits the merged tree whole, recording the rename (FileToCommit.OldName); the whole-tree commit also stops it carrying forward a file the merge deleted. The CLI prints RENAME lines where fossil's merge prints them. check-in: 2c3a78cf5e user: [email protected] tags: feat/horizon-cones, trunk

HZ-6.2-11: every reply to a request the origin classified carries the cursor; the operation-less round after a clone's last page answers no pull or clone card, so it is not a classified request, as in zit. HZ-6.2-12: a client records and sends the cursor whenever its transport names the origin; without one it announces in full, which the advisory cursor of §6.2.6 allows. check-in: a33cbbf951 user: [email protected] tags: feat/horizon-cones, trunk

Clone replies at a horizon carry the cursor and a horizon clone records it per origin (draft-fossil-horizon-00 §6.2.6), so the first exchange after a clone no longer announces the whole holding. Interoperates with zit both ways. check-in: 7e44f7107b user: [email protected] tags: feat/horizon-cones, trunk

The draft has an origin that classified a request emit the cursor in every reply, and a client that converged record the last reply's cursor with its artifact count per origin. A clone did neither, so the first exchange that sends after a horizon clone announced the whole holding.

A clone reply now carries the position it served the clone sequence through: below the sequence it hands on, or the origin's whole position once it hands on none. A horizon clone that names its origin records the last cursor and its own count, as zit does; the HTTP transport names it for the public Clone as for Sync. check-in: 7d9536347a user: [email protected] tags: feat/horizon-cones, trunk

Merge bases chosen as fossil's pivot_find chooses them: the most recent common ancestor by check-in time, with the check-out's pending merges as further secondaries, in place of a bidirectional breadth-first search that picked a different base on criss-cross and skewed histories. Checked against fossil 2.28 on 114 queries and a byte-for-byte merge. check-in: 97e188b6ef user: [email protected] tags: feat/horizon-cones, trunk

The ncruces driver reads event.mtime, declared DATETIME, as a time.Time, so the pivot search failed to scan its seed under -tags test_ncruces. coalesce() drops the declared type, as the parent query's already did. check-in: 89c6ae4aa6 user: [email protected] tags: feat/horizon-cones, trunk

The merge base is now fossil's pivot, so check it against fossil(1) on every run rather than against an expectation written once. Scripted graphs, loaded into both implementations, cover a skewed criss-cross (the newest common ancestor two edges away, not the one an edge away), a criss-cross at equal times in both load orders (the higher rid wins the tie), merges of merges and a query with two secondaries, and --ignore-merges. A merge of p into a checkout of s in the skewed graph writes the same bytes as fossil's merge and differs from a merge against the breadth-first base.

TestMergeBaseFossilOracle builds six seeded random graphs of sixteen check-ins, with equal times and children older than their parents, and compares 114 merge-base questions, both directions, with and without --ignore-merges and with two secondaries, against fossil merge-base. check-in: 944b44faf6 user: [email protected] tags: feat/horizon-cones, trunk

go-libfossil picked a merge base with a bidirectional breadth-first walk, the common ancestor nearest by edge count. fossil's merge uses pivot.c's pivot_find: a queue ordered by pending DESC, mtime DESC that expands the newest pending check-in first and returns the first one with a child queued from the other side, the most recent common ancestor by event time. The two differ on criss-cross merges and skewed histories, so a merge here could diff against another base than fossil's and write other bytes.

merge.FindPivot ports pivot_find in memory: (rid, side) rows with REPLACE re-pending, ties broken as a reverse scan of aqueue_idx1 yields them (rid, then side, descending), a missing event mtime counted as 0.0, ignoreMerges following primary links only, and the walk bound of the horizon merge-base query kept as ErrBound. FindCommonAncestor and FindCommonAncestorWithin now ask it.

Call sites: - Checkout.Merge takes fossil's pid: the merged-in version primary, the checkout's version and every vmerge id=0 merge the secondaries (merge.c 1076-1083), and gains Baseline, fossil's --baseline. The name pivot nid (pivot_find(1), merge.c 1089-1096) feeds only add_renames and the rename fill-in (merge.c 1192-1257); Merge does not follow renames, so it is not ported. Update and stash use the checkout version as their pivot and pick no base. - Repo.Merge (branch merge) makes the source branch tip the primary. - The origin's horizon-mergebase answer makes the merged-in check-in primary and declines to answer from a walk that reached history it does not hold. - Deepen's local deepen point keeps a boundary target among C's ancestors as its own point, since pivot_find seeds from check-ins only.

Horizon client rule (draft-fossil-horizon-00 §5.5, agreed for the draft): a common ancestor found in part of the graph need not be the one the whole graph gives, so a merge asks the origin whenever the walk reaches a check-in the store does not hold, not only when the base it finds is not held. MergeBaseCrossing takes that from Pivot.Unheld, and MergeContext merges over the origin's base as a baseline, deepening only when the store does not hold it.

Changed expectations: the horizon merge fixtures (hzStruct's y into feat2, structuralMergeSource, TestBranchMergeRefusesBaseBelowHorizon) fork f below the horizon, merge c2 into f's line and f into c2's: fossil picks c2, not f. On that shape fossil 2.28 answers fossil merge-base y x -> pivot=52d5923b8d (c2) with f=8ac24055 also a common ancestor. The fixtures now add a check-in on f's line newer than the horizon, which fossil does pick, to keep a base below the horizon, and the crossing tests check both the held-base walk that crosses and the base below. check-in: a23c7b12a9 user: [email protected] tags: feat/horizon-cones, trunk

Appendix A step 6: the merge-base query (pragma horizon-mergebase), deepen (Repo.DeepenHorizon, repo cone deepen), merges and bisect that refuse or deepen across the horizon (ErrBelowHorizon, Checkout.MergeContext), the origin's push-time check of a branch cancellation, seed verification by deepening (Repo.VerifySeed, repo verify --seed), and symbolic names below the horizon resolving. Interoperates with zit at 9bf7ebfe0f. check-in: 19d7db48d4 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.5: a tag applied to a check-in below the horizon resolves to that check-in's hash, because the control artifact is held. The resolver's sym-tag step joins event, which a check-in below the horizon has no row in, so the name was unknown and an operation on it never reached the horizon diagnostic. Both the tag: form and a plain name now fall back, in a structural store only, to the check-in below the horizon the tag names; Repo.ResolveVersion, Checkout.ResolveVersion and the CLI share the resolver, and checking it out names the horizon and the deepen. check-in: 67bb246d16 user: [email protected] tags: feat/horizon-cones, trunk

# Conflicts: # conformance/requirements/horizon.tsv check-in: 75ed88e8e9 user: [email protected] tags: feat/horizon-cones, trunk

Record Appendix A step 6 in the horizon requirements table and on the sparse page: the API and commands, what each costs, and what HZ-5.5-1 still leaves out (a symbolic name on a check-in below C, and annotate walking back past C). check-in: 078df932f4 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.5 has a bisection whose good end lies below the horizon deepen to it before it begins. The path between the ends ran through a boundary phantom and the session went on as if the history were there; Next now refuses first, naming the horizon and the deepen that brings the good end, matching content.ErrBelowHorizon. The library session holds no peer, so the deepen is left to the caller. check-in: ad60dffec8 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.9 asks for a verification mode so trusting the seed is a choice with a known price. sync.VerifySeed cuts the structural horizon to the content depth at the same C, which fetches every manifest below C and with them the real tag artifacts and primary-parent chains, discards the seed, and compares what the store then derives at each entry with what the seed asserted. Repo.VerifySeed and 'repo verify --seed' expose it; the store stays a content-horizon store afterward, and a failed transfer puts the structural cone and seed back. zit 9bf7ebfe0f offers no such mode. check-in: 35db744ea7 user: [email protected] tags: feat/horizon-cones, trunk

A check-in that creates a branch cancels its parent's branch tag by the name its clone derived, and under a structural horizon that name came from the seed. Section 5.7 has the origin check it against its own derivation: the handler now looks at each check-in a push stored, once crosslinked, and when the parent is on another branch here it rolls the whole push back with an error naming the stale seed and the parent's branch, spelled as zit's pushSeedCancelFault spells it. A check-in is admitted when the push or the store holds a control artifact cancelling the parent's real branch on it, and so is one whose parent this origin cannot place.

zit's check reads only the push for the correction; this one also reads the store, as the draft allows. Interop runs both ways against zit 9bf7ebfe0f. check-in: 8f89198ca1 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.5 forbids merging against a base the store does not hold. In a structural store the walk can reach a boundary check-in as the base, or find none where both sides' ancestry leaves the store, and the merge then failed on a phantom with an error that never named the horizon. content.MergeBaseCrossing decides the crossing for checkout merges and branch merges alike, and the error names the horizon, the base and the deepen, matching libfossil.ErrBelowHorizon.

Checkout.MergeContext is the remedy zit's merge applies: given a peer it asks the origin for the base, deepens to reach it and merges again over the real base. 'repo merge' takes the recovery flags and contacts the remote only when a merge crosses the horizon. check-in: d8458ad960 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.8 makes deepen the universal remedy of the draft: name a C' that is an ancestor of C and synchronize at it. sync.Deepen finds C' as zit's deepenPoint does, the target itself when it is C's ancestor, else a common ancestor over the P cards the store holds or asked of the origin with the merge-base query; it refuses rather than move C off its own ancestry when none can be established. Repo.DeepenHorizon and 'repo cone deepen' expose it, and the crossing diagnostic now names that command.

Moving a structural horizon through FetchCone now discards the held seed before the transfer, so the request offers none and the seed for C, applied at a later date, cannot shadow propagation from the seed for C'. A transfer that fails puts the old cone and seed back, as zit's deepen does: a store left structural without a seed derives no branch state for K. check-in: c11b3055fb user: [email protected] tags: feat/horizon-cones, trunk

A structural clone cannot compute a merge base whose ancestry lies below its horizon, so draft-fossil-horizon-00 section 6.2.4 lets it ask the origin with pragma horizon-mergebase A B. The origin answers in its pull reply, ahead of the igots where zit places it, with the base or - for no common ancestor, and only for check-ins it holds. Section 8 bounds the walk: merge.FindCommonAncestorWithin carries the held-set walk's visited-node bound and the query goes unanswered past it.

The client asks once, in the first request after the cursor, through SyncOpts.MergeBaseOf and reports the answer in the result. Interop runs both ways against zit 9bf7ebfe0f. check-in: 5612ac9771 user: [email protected] tags: feat/horizon-cones, trunk

History views render a labelled horizon boundary as the horizon (draft-fossil-horizon-00 §5.4): Repo.Ancestry ends there without an error, Repo.Timeline and LogEntry report it in HorizonParents, and repo timeline prints it as (horizon). An unlabelled absent parent renders as before. check-in: d0c44dbc25 user: [email protected] tags: feat/horizon-cones, trunk

A parent the origin labelled below a structural horizon is the horizon itself (draft-fossil-horizon-00 5.4), but the history views showed it as a bare parent hash, and the ancestry walk tried to read an event for it.

LogEntry and TimelineEntry gain HorizonParents, the subset of Parents that carries a boundary label; Ancestry ends at such a parent without an error; repo timeline prints it as "parent: <hash> (horizon)". An absent parent that is not labelled is transit and renders as before. check-in: ee3ccc7c67 user: [email protected] tags: feat/horizon-cones, trunk

The structural horizon (draft-fossil-horizon-00 Appendix A step 5): a store at structural depth elides manifests and nodes below the horizon and keeps boundary labels; the client sends horizon-structural and horizon-seed and holds the seed it is sent; the origin restricts manifests to the held set, labels the boundary, mints and sends the seed, and serves structural only when the seed is current. Interoperates with zit at 9bf7ebfe0f in both directions. check-in: 78ed31bf91 user: [email protected] tags: feat/horizon-cones, trunk

A docs page for sparse cones and horizons: concepts, the library API, the CLI, interop with zit and fossil 2.28, and the limits as they stand. check-in: 441604df52 user: [email protected] tags: feat/horizon-cones, trunk

The structural horizon's seed (draft-fossil-horizon-00 §5.2): an origin mints a control artifact carrying the tags in force at every entry of the held set, a clone holds it outside the artifact store where no sync, clone or rebuild can reach it, and the derivation applies it so every check-in in K derives as it does densely. Byte-identical to zit's seed when the latest summarized event is on a whole second. Conflicts in the horizon suite and table kept both sides; the seed's zit test now probes for a horizon-capable zit like the other horizon interop tests. check-in: 12dcbdeed6 user: [email protected] tags: feat/horizon-cones, trunk

deck.Marshal writes D to the millisecond, so a go-libfossil seed could never match zit's bytes or name even over identical state. Mint now writes D as YYYY-MM-DDTHH:MM:SS, keeping the round-up to the next whole second that Section 5.2.2's "no earlier than the latest tag event" needs, and recomputes Z over the body. Over whole-second tag events the seed is now byte-identical to zit's, which the interop test asserts for the full seed and its name. check-in: 038e50befb user: [email protected] tags: feat/horizon-cones, trunk

The cursor slice keeps a client's horizon cursor per origin, named by the internal SyncOpts.Origin, but the public Repo.Sync never set it, so a library or CLI sync kept no cursor and every request announced in full. The HTTP transport now names its peer by its URL with any password removed, as last-sync-url keeps it, and Sync passes that through. Other transports name no origin and keep announcing in full, which §6.2.6 allows. The test pulls with a credential-bearing URL and finds the cursor under the password-free one, and no config row holding the password. check-in: 331b540f1c user: [email protected] tags: feat/horizon-cones, trunk

The horizon cursor (draft-fossil-horizon-00 §6.2.6): an origin that classified a request answers pragma horizon-cursor with its position, a converged client records it per origin and cone and sends it back, and neither side announces again what the other already has. Interoperates with zit both ways. check-in: f704c6d58b user: [email protected] tags: feat/horizon-cones, trunk

A horizon pull announced the client's whole holding with igot on every request, and the origin answered with its whole kept inventory: one card per artifact, both ways, on every pull. draft-fossil-horizon-00 §6.2.6 replaces both with a position.

The origin values pragma horizon-cursor right after horizon-served in every reply to a horizon pull, as zit places it. Its position is rid order (the clone SEQNO sense), stopped short of the lowest phantom: a filled phantom keeps its old rid, so a cursor covering it would hide the arrival. Receiving a cursor, any reply that pulls announces nothing at or below it. A filled phantom that a check-in from an earlier receipt already named as parent is recorded as the reclassification position; a cursor below it is answered as if absent. A cursor beyond the origin's own position is ignored.

The client keeps, per origin and cone, the last reply's cursor and the rid count the origin holds through, sends the cursor on every request, announces nothing on a pull-only request and only what it appended on one that sends. Sessions name their origin with SyncOpts.Origin; without it nothing is kept and every request announces in full. check-in: b61bd3475c user: [email protected] tags: feat/horizon-cones, trunk

TestZitHorizonInterop took whatever zit was on PATH. A zit release from trunk reports the same version as the horizon-rfc build but has no --horizon flag, so the test failed in zit's argument parser. The new zitHorizonBinary probes the binary for horizon support, as zitBinary already probes for send-submanifests, and skips it, or fails the run under REQUIRE_ZIT_BIN=1, with the reason. check-in: 27b85364f2 user: [email protected] tags: feat/horizon-cones, trunk

The public horizon API in the shape approved: a horizon is part of Cone (NewHorizonCone, Cone.Horizon, Cone.WithHorizon), store cones carry it through SetStoreCone and SetStoreConeContext and the new CloneOpts.StoreCone, checkout cones refuse it with ErrHorizonCheckout, and the CLI takes --horizon and --leaves on repo clone and repo cone store and --no-horizon on repo cone store. check-in: 964b91f682 user: [email protected] tags: feat/horizon-cones, trunk

A store cone could carry a horizon only through internal/cone, so library users and the CLI had no way to clone into or edit a content-horizon store. Add Horizon, HorizonRule, NewHorizonCone, Cone.Horizon and Cone.WithHorizon over the internal cone (always the content cut), carry it through SetStoreCone/SetStoreConeContext (deepening fetches what the older check-in keeps) and CloneOpts.StoreCone, and add --horizon, --leaves, --no-horizon and --cone to repo clone and repo cone store.

A horizon is a property of the store, not of a checkout (draft-fossil-horizon-00 3.2), so checkout cones carrying one are refused with ErrHorizonCheckout, and widening the store from a checkout cone keeps the store's horizon. check-in: e5904fc85b user: [email protected] tags: feat/horizon-cones, trunk

Appendix A step 3 without the cursor: the client sends pragma horizon, the origin computes the held set over its dense graph, sends only the blobs above the horizon, inflates crossing deltas, sends no cluster, answers horizon-served content and refuses what it cannot cover (§6.4). Against a horizon-ignorant origin the client requests what its clusters name, as transit (§4.5 after zit#266). Interoperates with zit both ways. check-in: 84be5ad824 user: [email protected] tags: feat/horizon-cones, trunk

A structural-horizon clone lacks C's ancestry, so tag propagation has nowhere to start: every check-in in K would derive no branch, and closed markers and names set below C would vanish. The seed of draft-fossil-horizon-00 Section 5.2 restates the dense tag state at each entry of K so the unmodified derivation comes out exact.

internal/horizon/seed mints it at an origin (T cards per row in force at C and each check-in of K whose primary parent is outside K; D the latest summarized event rounded up to a whole second; U the origin's login), and holds it at a clone in a table of its own, outside the artifact store, so nothing that enumerates or serves artifacts can send it. It is applied when held, by Rebuild, and by seed.Apply, which the sync layer calls after each round. Replacing or discarding a seed re-derives the store, since only a re-derivation separates the old seed's rows from those real artifacts wrote. A Verifier hook carries the Section 5.2.4 provenance policy; no signing is implemented.

The crosslink-table clearing moves from verify into manifest.Rederive so Rebuild and the seed share one owner of re-derivation. The cards match zit's seed byte for byte in T and U, and each implementation derives from the other's seed what it derives from its own. check-in: 45fb2c7ed4 user: [email protected] tags: feat/horizon-cones, trunk

The Appendix A step 4 measurement: an opt-in corpus test reporting what a structural horizon saves over a content horizon at several depths. check-in: 5430186930 user: [email protected] tags: feat/horizon-cones, trunk

Opt-in corpus test (LIBFOSSIL_CORPUS) for draft-fossil-horizon-00 Appendix A step 4: for C at 10, 100 and 1000 check-ins back along trunk's primary-parent chain and at the check-in nearest a year before the tip, compare a content horizon's transfer with a structural horizon's under both held-set rules. It works on a copy of the repository. The seed is estimated as one control artifact with one T card per tag in force at each entry of K, uncompressed.

fossil-scm.org clone: 67894 artifacts, 37.2 MiB stored, 8425.2 MiB expanded (21008 manifests, 8.9 MiB stored, 1099.8 MiB expanded; no sub-manifest nodes; 6263 other structure, 691 clusters). Runtime 58s, peak heap 161 MiB. Share = (content - structural) / content.

descendants rule K below (n / MiB exp) content MiB exp structural MiB exp share exp share stored 10 back 12 20996 / 1098.9 1136.5 37.6 96.7% 49.0% 100 back 121 20887 / 1089.9 1262.0 172.1 86.4% 48.5% 1000 back 1746 19262 / 962.2 2341.0 1379.0 41.1% 40.0% 1 year (426 back) 796 20212 / 1036.1 1846.3 810.3 56.1% 44.5%

leaves rule: K = 20907 of 21008 at every depth; share 0.0% expanded, -7.7% stored (the seed outweighs the 101 manifests below). check-in: c6ef4af29f user: [email protected] tags: feat/horizon-cones, trunk

The re-pinned §4.5 (zit#266) splits the cluster rule by origin. An origin that says horizon-served announces what it keeps directly, so a horizon store must not request a name only a cluster gives it. A horizon-ignorant origin, fossil(1) among them, folds 100 or more unclustered artifacts into a cluster as it answers a pull and announces only the cluster, so a new check-in may be named nowhere else: the store must request what the cluster names, as transit, and classify what arrives. A horizon store acted on no cluster from either, and so never received such a check-in from fossil.

CrosslinkCluster now mints member phantoms in every store, as fossil does; the received-file filter already drops the cluster and the blobs below the horizon at the end of the transfer. Whether to act on a cluster is decided where the origin is known: once a reply of the session says horizon-served content, the sync client asks for no cluster catalog and the sync and clone clients store no cluster file card, so no cluster-only phantom is minted from such an origin.

A real fossil(1) case pulls a leaf announced only through the cluster fossil makes, and holds it and its blob without holding the cluster. check-in: 588d5b264e user: [email protected] tags: feat/horizon-cones, trunk

A horizon store asked every peer for the whole history and threw away what fell below C on arrival, so a horizon clone cost a dense clone's bandwidth. draft-fossil-horizon-00 §6 lets the request carry the horizon and the origin restrict what it sends.

The client sends `pragma horizon HASH RULE` after its path-cone pragmas on every clone and pull, never horizon-structural, and counts the replies that say `pragma horizon-served content` (HorizonServed on the clone and sync results, internal and public). It still classifies every reply locally, so a horizon-ignorant peer stays correct (§6.3).

The handler reads the pragma into the request's cone and serves it through the existing keep machinery rather than a parallel path: the pull inventory's keep set drops the blobs below the horizon (via ConeReferences.HorizonBelow) and every cluster; the clone walk walks the tree of a held-set check-in only, counts a parent as sent only inside the held set, and sends every sub-manifest node at its row while sending no cluster. A crossing delta is inflated through the same sourceKept test the path cone uses, charged to the round's batch. req-clusters under a horizon answers nothing, and every reply says horizon-served content.

A horizon store refuses, through refuseClone, a dense clone and a horizon whose held set is not inside its own, both computed over the store with horizon.Place. That is the spec's subset test; zit refuses any other C or rule. A path-only store now also refuses a horizon-only request, which keeps every path.

Interop with zit at horizon-rfc bbd1435674 holds both ways: zit clones and pulls at a horizon from the handler, the client from `zit serve`, and each side's horizon store refuses the other's dense clone. check-in: 352a0d9e0b user: [email protected] tags: feat/horizon-cones, trunk

zit#266 reported that a horizon clone pulling from stock Fossil never learns a check-in Fossil announced only inside a cluster: xfer.c clusters 100 or more unclustered artifacts and then announces only the cluster. §4.5 at 9bf7ebfe0f binds the cluster prohibition to an origin whose replies carry horizon-served, and has the clone request a horizon-ignorant origin's cluster members as transit and classify what arrives. HZ-4.5-1 is restated; implementing it waits for horizon-served, which the horizon pragmas bring. The sync draft on that branch differs from the trunk copy pinned here only by §9's editorial list of extension pragma names. check-in: d5926b325c user: [email protected] tags: feat/horizon-cones, trunk

A content-horizon store against any peer: below-horizon blobs it is sent are recorded as elided rather than stored, it holds and requests no cluster, and a below-horizon blob an operation needs is fetched on demand. check-in: 27b8439777 user: [email protected] tags: feat/horizon-cones, trunk

No peer speaks the horizon pragmas yet, so a content-horizon store receives whatever a dense peer sends (draft-fossil-horizon-00 §6.3). The received-content filter now applies the horizon's classification, read from internal/content rather than re-derived: blobs below the horizon become elided phantoms and are counted in the result's Elided. Blobs no held manifest names yet stay as unreferenced holdings, and retained delta bases stay transit (§4.2).

A horizon store also acts on no cluster (§4.5). Crosslinking a cluster applies its tag but makes no member phantoms. The filter drops the cluster's content, and classification elides any phantom that carries the cluster tag. The client stops sending req-clusters. Path-cone-only stores keep today's cluster behaviour.

Conformance cases run against go-libfossil's handler and fossil(1): clone and pull, deepen, demand fetch through UpdateContext, transit deltas, and cluster rules. check-in: f1f4240f40 user: [email protected] tags: feat/horizon-cones, trunk

Sparse-store classification sized to what it proves rather than to the store: file content is never read, completely linked check-ins answer from the crosslink tables, and a content horizon reads only its held set's trees. A path-cone store of fossil-scm.org classifies in about 2 s at a 65 MiB peak, where v0.14.0 failed with a byte-budget error. check-in: cd1b25e114 user: [email protected] tags: feat/horizon-cones, trunk

Sparse classification read, inflated and delta-expanded every held blob, file content included, under a whole-store byte budget. On a clone of the Fossil SCM repository (67,894 blobs, 8.4 GB expanded) LoadAbsences failed at once with "byte budget exceeded", so no path-cone or content-horizon store of a real repository could classify its phantoms, and pull, verify and compaction failed with it.

Reference facts now come from the tables wherever they provably stand in for raw cards. A blob the crosslinker recorded as no artifact and mlink names as a file version is never read. A completely linked check-in (event row, no orphan row, a primary parent edge) whose primary parent and baseline trees are themselves reproduced contributes its mlink rows as F cards and its plink, cherrypick and tagxref rows as references; by induction over the primary chain those rows give the same (name, file) pairs as its raw cards. Everything else is still read raw: unlinked and orphaned check-ins, roots, check-ins whose parent never arrived, other artifacts, and every check-in once the store has seen a sub-manifest, since trees are not crosslinked. Budgets are charged per round for the selected blobs and their delta ancestors only, and a base's bytes are released once its last delta is applied.

A content horizon now reads only its held set's manifests and the baselines and nodes they reach, on demand, with one visited set and a byte budget scaled to the held set.

On the Fossil SCM clone, cone {src} classifies in about 2 s at a 65 MiB peak heap, reading 8,655 of 67,894 blobs; the inside/outside sets equal the all-raw result, and Outside/Required agree on every artifact. check-in: b28c06506b user: [email protected] tags: feat/horizon-cones, trunk

The content horizon's store classification: a blob is elided when no check-in in the held set names it through its full tree, a kept blob is in both keep sets under a path cone, structure is never elided under a horizon, and a horizon the store cannot place elides nothing, as zit does. check-in: bc54a61b2c user: [email protected] tags: feat/horizon-cones, trunk

# Conflicts: # conformance/horizon_test.go # conformance/requirements/horizon.tsv check-in: 80c8df6a74 user: [email protected] tags: feat/horizon-cones, trunk

A store cone may now carry a content horizon (draft-fossil-horizon-00 §3.2, §4.1-§4.3). Classification computes the held set K from the store's own check-in graph and walks K's full trees -- F cards directly, through B baselines and through S sub-manifest nodes, with one visited set over structural names -- because mlink records a file version only where it changed, and a blob introduced below C and unchanged through C is named by C's tree alone. Nothing about K is stored: every classification recomputes it, so moving C or losing the cone is a reload, never a migration.

Blobs no K check-in names are elided; with path members as well a blob is kept only when it is in both keep sets. Under a horizon nothing structural is elided, by the horizon or by path members, and delta bases of held artifacts and shunned names keep their classes. A C the store does not hold elides nothing, as zit's classification does: eliding everything would leave the pull that brings a deepened C nothing to ask for. A structural horizon is refused by SetStoreCone until it is implemented.

internal/content now imports internal/horizon, so the held-set tests, which commit through internal/manifest, move to an external test package to avoid an import cycle. check-in: 2ea6479ba9 user: [email protected] tags: feat/horizon-cones, trunk

zit's author called horizon-rfc bbd1435674 ready to implement against, and zit#265 settled the leaves rule there against fossil's leaf: a childless check-in is one no P card names, open or closed, and K under the leaves rule is C's descendants plus every descendant of each check-in where a childless check-in's ancestry meets C's. Both rules now yield a set closed under descendants, so leaves and branch tips derive exactly.

HeldSet follows zit's computation at that check-in (src/sparse.zig, ClassifyState.extend): meeting points are the members of C's ancestry a childless check-in's parent walk reaches without passing one, and a childless check-in is skipped only when it descends from C. The tests that encoded the old rule are restated: a merged-but-open branch tip adds nothing, a closed branch's tip does, the fork's descendants on C's side are held, and an unrelated root adds nothing.

The draft copy, the README pin and the table follow: HZ-3.1-1, HZ-5.2-1 (the seed applies to every entry of K) and HZ-6.2-11 (announce only in reply to a request that pulls) are restated, and HZ-3.1-4 (closure under descendants) is new, with a case checking that no child of a held check-in falls outside K under either rule. check-in: a68f89e414 user: [email protected] tags: feat/horizon-cones, trunk

Both slices defined the descendants/leaves rule: cone.HorizonRule for the serialized member and horizon.Rule for the walk. One concept gets one type; the walk now takes the value the cone carries. check-in: 6d8ccec5da user: [email protected] tags: feat/horizon-cones, trunk

The held set K of draft-fossil-horizon-00 §3.1 under both rules, bounded and computed from the check-in graph the repository holds. Conflicts in the horizon suite and table resolved by keeping both slices' cases and rows. check-in: 7a5ae7cee7 user: [email protected] tags: feat/horizon-cones, trunk

draft-fossil-horizon-00 Section 3.1 defines K as a function of C, the rule and the check-in graph, recomputed by whoever needs it and stored by no one. internal/horizon.HeldSet computes it from the plink and leaf tables: one bounded plink load, then iterative breadth-first walks.

Merge parents are followed both ways, as zit's walk does (src/sparse.zig, ClassifyState.extend and walkParents). Under the leaves rule each open leaf's ancestry is held up to where it meets C's ancestry, inclusive, so every equally near common ancestor the walk reaches is held. "Open leaf" is fossil's leaf table minus active closed tags rather than zit's "no held child", which keeps a below-C branch merged into a descendant held back to its fork, so any two open leaves keep their merge base in K. C that is not a held check-in is ErrNotHeld; phantom parents are never entered.

Section 8 requires the walk to carry visited-node bounds: the load and every walk share the 1<<20 bound of the other graph walks and fail with ErrBound, and cancellation is honoured. The conformance suite gains cases for HZ-3.1-1 and HZ-3.1-3, both PARTIAL until the horizon member and the content horizon exist. check-in: 75ded1b0e0 user: [email protected] tags: feat/horizon-cones, trunk

The horizon member of a cone and its version-3 serialization, byte-identical to zit's (draft-fossil-horizon-00 §3.3). check-in: 6da46813f9 user: [email protected] tags: feat/horizon-cones, trunk

A horizon names one check-in by full hash, a held-set rule and a depth. Version 3 carries it between the count and the flagged members, a distinct version so a reader predating horizons refuses the cone instead of reading the lead-in as path members. Forms without a horizon are unchanged, so their digests hold. A horizon-only cone has no path members and is path-dense. Parse re-serializes and compares, so an upper-case hash or any other non-canonical input is refused.

HZ-3.3-1 and HZ-3.3-2 are now covered. check-in: 5f637a3b62 user: [email protected] tags: feat/horizon-cones, trunk

Moves the horizon copy from 86d4b67372 to e6ee276c89, the check-in the sparse-cone copy is already pinned at. Every change between the two is in §6: §6.1 lets any request carry pragma horizon-cursor, §6.2 adds the SHOULDs to keep the classification and the structural graph per store prefix, and §6.2.6 defines the cursor. Five new rows read them into the table (HZ-6.2-9 to HZ-6.2-13) and HZ-6.1-1 names the cursor; the table is now 50 requirements, none implemented. check-in: be103fe35a user: [email protected] tags: feat/horizon-cones, trunk

chore(release): v0.14.0 changelog leaf check-in: 6d5357235a user: [email protected] tags: main, trunk, v0.14.0

feat: sub-manifest trees and sparse cones (draft-fossil-submanifests-00, draft-fossil-sparse-cone-00) check-in: 7244496a09 user: [email protected] tags: main, trunk

StartServer kills the server's process group and reaps the leader, but fossil server forks a child per connection, and SIGKILL reaches that child asynchronously. On a loaded machine the child could still finish a write after Wait returned, and t.TempDir's RemoveAll failed with "directory not empty" (TestCloneFromRealFossilUncompressedReply, load average ~20). This is the race #352 narrowed; cleanup now also waits, for at most five seconds, until the group has no member left. leaf check-in: f1b6155c8f user: [email protected] tags: feat/submanifest-reader, main, trunk

Five fixes the three-tool round trip and the fossil-scm.org comparison surfaced: an uncovered clone is refused with the push card first, repo status reports the checkout's changes, repo clone remembers last-sync-url, repo verify reports damage, transit phantoms and elision, and adding a directory adds the files beneath it. check-in: b3c5ad7bca user: [email protected] tags: feat/submanifest-reader, main, trunk

TestScenarioClusterConvergence calls content.GenerateClusters once the simulation is over, to check the API on a repository of 500 blobs. Buggify was still armed, so at hostile seeds 11 and 23 a write-path site corrupted a cluster artifact, blob.Store's round-trip check refused it, and the test failed on the refusal. That refusal is the safeguard working. The call now runs with injection suspended, as the invariant checks and fixtures do. check-in: 0be3dddeae user: [email protected] tags: feat/submanifest-reader, main, trunk

Repo.Verify expanded every non-phantom blob on its own with content.Verify. Each expansion re-applies the blob's whole delta chain, so the cost grows with the square of chain length. On a clone of fossil-scm.org's repository (67,894 blobs, chains averaging 532 deltas and reaching 2,546) repo verify took 49 minutes of CPU on both main and this branch, against six and a half minutes for fossil test-integrity.

content.ExpandAll already walks the delta forest from its roots and applies each delta once, binding every result to its name; internal/verify uses it. Verify now goes through it, with the same blob selection (size >= 0), and names the failing rid in its error. The same clone verifies in 19 seconds. check-in: 6f269e32a9 user: [email protected] tags: feat/submanifest-reader, main, trunk

The simulator arms Buggify when it is built, and the scenarios seed their fixtures after that. Two setups dropped blob.Store's error, so a store the injection failed left an empty UUID, and TestScenarioAuthRestrictedCaps panicked in blob.Exists's precondition when it asserted on it. The seed sweep hit this at adversarial seeds 5 and 8.

seedTx suspends injection for the fixture transaction and fails the test if it does not commit. Every unchecked setup transaction in the package now goes through it. check-in: c796b51895 user: [email protected] tags: feat/submanifest-reader, main, trunk

Manage stored a named directory, "." included, as a tracked file, and the next commit failed reading it. A named directory now expands through the checkout's bounded, cone-aware walk to the files beneath it, skipping paths with a dot component as fossil's add does without --dotfiles; a file named explicitly is kept as given. An uncovered path is refused as such before its absence on disk is reported. check-in: 9e28f473cc user: [email protected] tags: feat/submanifest-reader, main, trunk

repo verify hashed the held blobs and printed a fixed line. It now runs the full verifier the library uses for rebuild, prints each issue, and reports transit phantoms apart from elided artifacts, as sparse-cone 5.2 requires of a verification operation: a store complete within its cone verifies healthy and shows its elided count. check-in: 168062ca33 user: [email protected] tags: feat/submanifest-reader, main, trunk