Timeline

Rebuild of fossil-scm.org's repository in about 50 s instead of 3 to 6 minutes: delta-chain availability answered from the sweep's cache, and each manifest parsed once. Every derived table is identical before and after. check-in: 6339d6fb50 user: [email protected] tags: feat/horizon-cones, trunk

Rebuilding a clone of the Fossil SCM repository (67,894 artifacts, 21,008 check-ins) took about 3 minutes of which 104 s was parentTreeGap asking content.AvailableByUUID, an uncached delta-chain walk with one SQL statement per node, for every parent of every check-in. It now uses the sweep's availability cache like the rest of the guard.

The same sweep parsed each check-in three times (as a child, as its child's parent, as a delta baseline) and rebuilt its file maps. The sweep's content cache now keeps the parsed deck, file views and baseline rid of its few most recent check-ins (Cache.Derived), dropped with the cached bytes on Invalidate and Clear. Derived hits count as reuse in the cascade stats, so the shared-cache regression guard keeps measuring it.

The rebuilt event, mlink, plink, tagxref, leaf, filename, cherrypick, orphan, unclustered, unsent, forumpost, attachment, phantom, blob and delta tables are row-for-row identical to the previous code's. check-in: 48ff1f5084 user: [email protected] tags: feat/horizon-cones, trunk

The rebuild budget is a wall-time and memory target, and the test only reported time. Sampling HeapInuse beside the run makes a regression in either visible from one -v run. check-in: 7cb4d1aa0d user: [email protected] tags: feat/horizon-cones, trunk

Merges follow renames as fossil's merge does: its name pivot, find_filename_changes and file matching ported, so a file renamed on one side and edited on the other merges into the renamed file. Matches fossil 2.28 on 13 rename scenarios. Repo.Merge no longer brings back files the merge deleted. check-in: 4bccd3f803 user: [email protected] tags: feat/horizon-cones, trunk

A file renamed on one side and edited on the other was merged as a deletion plus an addition: the edit was kept under the old name and the renamed copy arrived beside it, so fossil and go-libfossil left different trees and recorded different changes for the same merge.

Port fossil's rename matching (merge.c merge_cmd): the name pivot nid (pivot_find(1) of M and V, rerun against P when they differ), the renames from nid to V, P and M (path.c find_filename_changes, one way, now path.FilenameChanges), and the fv table those fill (add_renames, the nid!=pid name fill-in, then the files of V, P and M) as merge.MatchFiles/FollowRenames. Checkout.Merge classifies each matched file as before and renames on disk and in vfile a file the merged-in version renamed and the checkout did not, after the other steps, moving swapped names together and setting aside a removed or deleted row that held the new name. Where fossil would rename over a kept tracked file or an untracked one, Merge refuses, as it already does for adds.

Repo.Merge matches files the same way and commits the merged tree whole, recording the rename (FileToCommit.OldName); the whole-tree commit also stops it carrying forward a file the merge deleted. The CLI prints RENAME lines where fossil's merge prints them. check-in: 2c3a78cf5e user: [email protected] tags: feat/horizon-cones, trunk

HZ-6.2-11: every reply to a request the origin classified carries the cursor; the operation-less round after a clone's last page answers no pull or clone card, so it is not a classified request, as in zit. HZ-6.2-12: a client records and sends the cursor whenever its transport names the origin; without one it announces in full, which the advisory cursor of §6.2.6 allows. check-in: a33cbbf951 user: [email protected] tags: feat/horizon-cones, trunk

Clone replies at a horizon carry the cursor and a horizon clone records it per origin (draft-fossil-horizon-00 §6.2.6), so the first exchange after a clone no longer announces the whole holding. Interoperates with zit both ways. check-in: 7e44f7107b user: [email protected] tags: feat/horizon-cones, trunk

The draft has an origin that classified a request emit the cursor in every reply, and a client that converged record the last reply's cursor with its artifact count per origin. A clone did neither, so the first exchange that sends after a horizon clone announced the whole holding.

A clone reply now carries the position it served the clone sequence through: below the sequence it hands on, or the origin's whole position once it hands on none. A horizon clone that names its origin records the last cursor and its own count, as zit does; the HTTP transport names it for the public Clone as for Sync. check-in: 7d9536347a user: [email protected] tags: feat/horizon-cones, trunk

Merge bases chosen as fossil's pivot_find chooses them: the most recent common ancestor by check-in time, with the check-out's pending merges as further secondaries, in place of a bidirectional breadth-first search that picked a different base on criss-cross and skewed histories. Checked against fossil 2.28 on 114 queries and a byte-for-byte merge. check-in: 97e188b6ef user: [email protected] tags: feat/horizon-cones, trunk

The ncruces driver reads event.mtime, declared DATETIME, as a time.Time, so the pivot search failed to scan its seed under -tags test_ncruces. coalesce() drops the declared type, as the parent query's already did. check-in: 89c6ae4aa6 user: [email protected] tags: feat/horizon-cones, trunk

The merge base is now fossil's pivot, so check it against fossil(1) on every run rather than against an expectation written once. Scripted graphs, loaded into both implementations, cover a skewed criss-cross (the newest common ancestor two edges away, not the one an edge away), a criss-cross at equal times in both load orders (the higher rid wins the tie), merges of merges and a query with two secondaries, and --ignore-merges. A merge of p into a checkout of s in the skewed graph writes the same bytes as fossil's merge and differs from a merge against the breadth-first base.

TestMergeBaseFossilOracle builds six seeded random graphs of sixteen check-ins, with equal times and children older than their parents, and compares 114 merge-base questions, both directions, with and without --ignore-merges and with two secondaries, against fossil merge-base. check-in: 944b44faf6 user: [email protected] tags: feat/horizon-cones, trunk

go-libfossil picked a merge base with a bidirectional breadth-first walk, the common ancestor nearest by edge count. fossil's merge uses pivot.c's pivot_find: a queue ordered by pending DESC, mtime DESC that expands the newest pending check-in first and returns the first one with a child queued from the other side, the most recent common ancestor by event time. The two differ on criss-cross merges and skewed histories, so a merge here could diff against another base than fossil's and write other bytes.

merge.FindPivot ports pivot_find in memory: (rid, side) rows with REPLACE re-pending, ties broken as a reverse scan of aqueue_idx1 yields them (rid, then side, descending), a missing event mtime counted as 0.0, ignoreMerges following primary links only, and the walk bound of the horizon merge-base query kept as ErrBound. FindCommonAncestor and FindCommonAncestorWithin now ask it.

Call sites: - Checkout.Merge takes fossil's pid: the merged-in version primary, the checkout's version and every vmerge id=0 merge the secondaries (merge.c 1076-1083), and gains Baseline, fossil's --baseline. The name pivot nid (pivot_find(1), merge.c 1089-1096) feeds only add_renames and the rename fill-in (merge.c 1192-1257); Merge does not follow renames, so it is not ported. Update and stash use the checkout version as their pivot and pick no base. - Repo.Merge (branch merge) makes the source branch tip the primary. - The origin's horizon-mergebase answer makes the merged-in check-in primary and declines to answer from a walk that reached history it does not hold. - Deepen's local deepen point keeps a boundary target among C's ancestors as its own point, since pivot_find seeds from check-ins only.

Horizon client rule (draft-fossil-horizon-00 §5.5, agreed for the draft): a common ancestor found in part of the graph need not be the one the whole graph gives, so a merge asks the origin whenever the walk reaches a check-in the store does not hold, not only when the base it finds is not held. MergeBaseCrossing takes that from Pivot.Unheld, and MergeContext merges over the origin's base as a baseline, deepening only when the store does not hold it.

Changed expectations: the horizon merge fixtures (hzStruct's y into feat2, structuralMergeSource, TestBranchMergeRefusesBaseBelowHorizon) fork f below the horizon, merge c2 into f's line and f into c2's: fossil picks c2, not f. On that shape fossil 2.28 answers fossil merge-base y x -> pivot=52d5923b8d (c2) with f=8ac24055 also a common ancestor. The fixtures now add a check-in on f's line newer than the horizon, which fossil does pick, to keep a base below the horizon, and the crossing tests check both the held-base walk that crosses and the base below. check-in: a23c7b12a9 user: [email protected] tags: feat/horizon-cones, trunk

Appendix A step 6: the merge-base query (pragma horizon-mergebase), deepen (Repo.DeepenHorizon, repo cone deepen), merges and bisect that refuse or deepen across the horizon (ErrBelowHorizon, Checkout.MergeContext), the origin's push-time check of a branch cancellation, seed verification by deepening (Repo.VerifySeed, repo verify --seed), and symbolic names below the horizon resolving. Interoperates with zit at 9bf7ebfe0f. check-in: 19d7db48d4 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.5: a tag applied to a check-in below the horizon resolves to that check-in's hash, because the control artifact is held. The resolver's sym-tag step joins event, which a check-in below the horizon has no row in, so the name was unknown and an operation on it never reached the horizon diagnostic. Both the tag: form and a plain name now fall back, in a structural store only, to the check-in below the horizon the tag names; Repo.ResolveVersion, Checkout.ResolveVersion and the CLI share the resolver, and checking it out names the horizon and the deepen. check-in: 67bb246d16 user: [email protected] tags: feat/horizon-cones, trunk

# Conflicts: # conformance/requirements/horizon.tsv check-in: 75ed88e8e9 user: [email protected] tags: feat/horizon-cones, trunk

Record Appendix A step 6 in the horizon requirements table and on the sparse page: the API and commands, what each costs, and what HZ-5.5-1 still leaves out (a symbolic name on a check-in below C, and annotate walking back past C). check-in: 078df932f4 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.5 has a bisection whose good end lies below the horizon deepen to it before it begins. The path between the ends ran through a boundary phantom and the session went on as if the history were there; Next now refuses first, naming the horizon and the deepen that brings the good end, matching content.ErrBelowHorizon. The library session holds no peer, so the deepen is left to the caller. check-in: ad60dffec8 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.9 asks for a verification mode so trusting the seed is a choice with a known price. sync.VerifySeed cuts the structural horizon to the content depth at the same C, which fetches every manifest below C and with them the real tag artifacts and primary-parent chains, discards the seed, and compares what the store then derives at each entry with what the seed asserted. Repo.VerifySeed and 'repo verify --seed' expose it; the store stays a content-horizon store afterward, and a failed transfer puts the structural cone and seed back. zit 9bf7ebfe0f offers no such mode. check-in: 35db744ea7 user: [email protected] tags: feat/horizon-cones, trunk

A check-in that creates a branch cancels its parent's branch tag by the name its clone derived, and under a structural horizon that name came from the seed. Section 5.7 has the origin check it against its own derivation: the handler now looks at each check-in a push stored, once crosslinked, and when the parent is on another branch here it rolls the whole push back with an error naming the stale seed and the parent's branch, spelled as zit's pushSeedCancelFault spells it. A check-in is admitted when the push or the store holds a control artifact cancelling the parent's real branch on it, and so is one whose parent this origin cannot place.

zit's check reads only the push for the correction; this one also reads the store, as the draft allows. Interop runs both ways against zit 9bf7ebfe0f. check-in: 8f89198ca1 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.5 forbids merging against a base the store does not hold. In a structural store the walk can reach a boundary check-in as the base, or find none where both sides' ancestry leaves the store, and the merge then failed on a phantom with an error that never named the horizon. content.MergeBaseCrossing decides the crossing for checkout merges and branch merges alike, and the error names the horizon, the base and the deepen, matching libfossil.ErrBelowHorizon.

Checkout.MergeContext is the remedy zit's merge applies: given a peer it asks the origin for the base, deepens to reach it and merges again over the real base. 'repo merge' takes the recovery flags and contacts the remote only when a merge crosses the horizon. check-in: d8458ad960 user: [email protected] tags: feat/horizon-cones, trunk

Section 5.8 makes deepen the universal remedy of the draft: name a C' that is an ancestor of C and synchronize at it. sync.Deepen finds C' as zit's deepenPoint does, the target itself when it is C's ancestor, else a common ancestor over the P cards the store holds or asked of the origin with the merge-base query; it refuses rather than move C off its own ancestry when none can be established. Repo.DeepenHorizon and 'repo cone deepen' expose it, and the crossing diagnostic now names that command.

Moving a structural horizon through FetchCone now discards the held seed before the transfer, so the request offers none and the seed for C, applied at a later date, cannot shadow propagation from the seed for C'. A transfer that fails puts the old cone and seed back, as zit's deepen does: a store left structural without a seed derives no branch state for K. check-in: c11b3055fb user: [email protected] tags: feat/horizon-cones, trunk

A structural clone cannot compute a merge base whose ancestry lies below its horizon, so draft-fossil-horizon-00 section 6.2.4 lets it ask the origin with pragma horizon-mergebase A B. The origin answers in its pull reply, ahead of the igots where zit places it, with the base or - for no common ancestor, and only for check-ins it holds. Section 8 bounds the walk: merge.FindCommonAncestorWithin carries the held-set walk's visited-node bound and the query goes unanswered past it.

The client asks once, in the first request after the cursor, through SyncOpts.MergeBaseOf and reports the answer in the result. Interop runs both ways against zit 9bf7ebfe0f. check-in: 5612ac9771 user: [email protected] tags: feat/horizon-cones, trunk

History views render a labelled horizon boundary as the horizon (draft-fossil-horizon-00 §5.4): Repo.Ancestry ends there without an error, Repo.Timeline and LogEntry report it in HorizonParents, and repo timeline prints it as (horizon). An unlabelled absent parent renders as before. check-in: d0c44dbc25 user: [email protected] tags: feat/horizon-cones, trunk

A parent the origin labelled below a structural horizon is the horizon itself (draft-fossil-horizon-00 5.4), but the history views showed it as a bare parent hash, and the ancestry walk tried to read an event for it.

LogEntry and TimelineEntry gain HorizonParents, the subset of Parents that carries a boundary label; Ancestry ends at such a parent without an error; repo timeline prints it as "parent: <hash> (horizon)". An absent parent that is not labelled is transit and renders as before. check-in: ee3ccc7c67 user: [email protected] tags: feat/horizon-cones, trunk

The structural horizon (draft-fossil-horizon-00 Appendix A step 5): a store at structural depth elides manifests and nodes below the horizon and keeps boundary labels; the client sends horizon-structural and horizon-seed and holds the seed it is sent; the origin restricts manifests to the held set, labels the boundary, mints and sends the seed, and serves structural only when the seed is current. Interoperates with zit at 9bf7ebfe0f in both directions. check-in: 78ed31bf91 user: [email protected] tags: feat/horizon-cones, trunk

A push alone carries no horizon pragma, so its reply says nothing about how the horizon is served, and a structural store pushing to its origin fell back to the content horizon. The first reply decides only when the request pulled. Preconditions of the structural serving and receiving paths are asserted. check-in: 1abe24cd2b user: [email protected] tags: feat/horizon-cones, trunk

A structural store elides a phantom nothing it holds names, and an igot makes exactly such a phantom, so a structural store serving a push never asked for the pushed check-in and a pull relied on the session alone to want it. A name the peer announced in the exchange is now asked for when nothing held names it, on both sides; a labelled boundary or a tag's below-horizon target stays elided. check-in: 26aff82ccc user: [email protected] tags: feat/horizon-cones, trunk

Composed with path members, a structural horizon keeps K's trees, but an origin may withhold their nodes outside the cone, as zit's does since 2b635ea5ce. Those nodes classified transit, so a clone from zit stalled waiting on them. A node the path members exclude is now elided under a structural horizon, as a path cone elides it; zit interop under cone src proves it both ways. check-in: 74ac2ea4c1 user: [email protected] tags: feat/horizon-cones, trunk

Cases for the ordinary commit from a structural store (§5.7), the leaf verdict over one (§3.1), verification and crossing (§5.4, §5.5), and the requirements table updated to what the structural horizon now covers; what remains, with the Appendix A step that covers it, stays in the notes. check-in: d9e74912e8 user: [email protected] tags: feat/horizon-cones, trunk

Horizon gains Structural, whose zero value is the content horizon, so a Horizon built before it means what it meant; clone and sync results say how many replies served the structural level and whether the store fell back. repo clone and repo cone store take --structural with --horizon.

A structural store's history ends at its horizon, which the commands now say rather than leave bare phantoms (draft-fossil-horizon-00 §5.4, §5.5): repo info names the horizon and the check-ins below it, repo verify counts them apart from damage, and checkout, diff and annotate of a check-in below C fail naming the horizon and the deepen that reaches it. check-in: c61a17d9ab user: [email protected] tags: feat/horizon-cones, trunk

zit clones and pulls at a structural horizon from go-libfossil's origin, and go-libfossil clones and pulls at one from zit serve. Each origin withholds the manifests below the horizon, labels the boundary and sends the seed; each client stays structural and derives the branch, leaf and tag state of the held set as a dense clone of the same origin does, including after a tag below the horizon moves the state at an entry and re-seeds it. zit holds the very seed go-libfossil mints. check-in: b76863738f user: [email protected] tags: feat/horizon-cones, trunk

The origin now answers pragma horizon-structural (draft-fossil-horizon-00 §6.1, §6.2): it keeps only the manifests and nodes the held set reaches, labels every boundary hash at the head of each reply (§6.2.2), mints the seed for C and sends it, pragma ahead of its file card, when the client offers another name (§6.2.3), and advertises the structural level only then; a request it cannot place or seed is served as content. It refuses a seed offered to it and withholds a gimme'd manifest below the horizon.

A structural store's requests carry horizon-structural and the seed it holds. It routes the announced card to seed.Hold after checking its name and shape, never storing or announcing it, records the labels, and applies the seed after each reply and after the closing link. A first reply that does not serve structural with a seed, or a seed failing provenance, makes it a content-horizon store for good, reported in the result, and the manifests below the horizon are fetched (§5.1, §5.2.4, §6.3). check-in: dc33bca3a6 user: [email protected] tags: feat/horizon-cones, trunk

A sparse store defers a check-in whose parent it lacks, so that mlinks wait on a complete parent tree. Under a structural horizon (draft -fossil-horizon-00 §5.1) the parents below the horizon never arrive, so C and every entry of K stayed unlinked. An absent source no longer defers there; the check-in links against the phantom as a dense store links it. check-in: 79ec72f88e user: [email protected] tags: feat/horizon-cones, trunk

The seed's rows name their target as their source, and the crosslink treats a blob that sources a tagxref row as already linked. A sync stores an entry before it links it, so a seed applied in between left the entry unlinked for good: no event, no plink, no leaf. Apply now skips a target the store has not linked; the sync layer applies again after the round that links it. check-in: c7c0c130bd user: [email protected] tags: feat/horizon-cones, trunk

A structural-horizon clone (draft-fossil-horizon-00 §5.1) keeps only the manifests and nodes its held set reaches, so its phantoms need classes the content horizon never had: a parent the origin labelled below the horizon is elided, an unlabelled one stays transit because it may only be late (§5.3), and a phantom only a tag or cherry-pick names is a manifest below the horizon nothing in K reaches. SetStoreCone stops refusing the depth, and keeps the boundary labels beside the store cone, dropping them when the horizon moves since they name what lies below one C. check-in: 31b140507a user: [email protected] tags: feat/horizon-cones, trunk

A docs page for sparse cones and horizons: concepts, the library API, the CLI, interop with zit and fossil 2.28, and the limits as they stand. check-in: 441604df52 user: [email protected] tags: feat/horizon-cones, trunk

The page described the leaves rule as each childless check-in's ancestry back to its meeting point with C, the rule before zit#265. The rule at the pinned draft holds every descendant of each meeting point, which is why it holds nearly all of fossil-scm.org's history: old unmerged branches fork near the root. check-in: 097153487d user: [email protected] tags: feat/horizon-cones, trunk

The structural horizon's seed (draft-fossil-horizon-00 §5.2): an origin mints a control artifact carrying the tags in force at every entry of the held set, a clone holds it outside the artifact store where no sync, clone or rebuild can reach it, and the derivation applies it so every check-in in K derives as it does densely. Byte-identical to zit's seed when the latest summarized event is on a whole second. Conflicts in the horizon suite and table kept both sides; the seed's zit test now probes for a horizon-capable zit like the other horizon interop tests. check-in: 12dcbdeed6 user: [email protected] tags: feat/horizon-cones, trunk

The cursor slice keeps a client's horizon cursor per origin, named by the internal SyncOpts.Origin, but the public Repo.Sync never set it, so a library or CLI sync kept no cursor and every request announced in full. The HTTP transport now names its peer by its URL with any password removed, as last-sync-url keeps it, and Sync passes that through. Other transports name no origin and keep announcing in full, which §6.2.6 allows. The test pulls with a credential-bearing URL and finds the cursor under the password-free one, and no config row holding the password. check-in: 331b540f1c user: [email protected] tags: feat/horizon-cones, trunk