Timeline

The structural horizon's seed (draft-fossil-horizon-00 §5.2): an origin mints a control artifact carrying the tags in force at every entry of the held set, a clone holds it outside the artifact store where no sync, clone or rebuild can reach it, and the derivation applies it so every check-in in K derives as it does densely. Byte-identical to zit's seed when the latest summarized event is on a whole second. Conflicts in the horizon suite and table kept both sides; the seed's zit test now probes for a horizon-capable zit like the other horizon interop tests. check-in: 12dcbdeed6 user: [email protected] tags: feat/horizon-cones, trunk

deck.Marshal writes D to the millisecond, so a go-libfossil seed could never match zit's bytes or name even over identical state. Mint now writes D as YYYY-MM-DDTHH:MM:SS, keeping the round-up to the next whole second that Section 5.2.2's "no earlier than the latest tag event" needs, and recomputes Z over the body. Over whole-second tag events the seed is now byte-identical to zit's, which the interop test asserts for the full seed and its name. check-in: 038e50befb user: [email protected] tags: feat/horizon-cones, trunk

The cursor slice keeps a client's horizon cursor per origin, named by the internal SyncOpts.Origin, but the public Repo.Sync never set it, so a library or CLI sync kept no cursor and every request announced in full. The HTTP transport now names its peer by its URL with any password removed, as last-sync-url keeps it, and Sync passes that through. Other transports name no origin and keep announcing in full, which §6.2.6 allows. The test pulls with a credential-bearing URL and finds the cursor under the password-free one, and no config row holding the password. check-in: 331b540f1c user: [email protected] tags: feat/horizon-cones, trunk

The horizon cursor (draft-fossil-horizon-00 §6.2.6): an origin that classified a request answers pragma horizon-cursor with its position, a converged client records it per origin and cone and sends it back, and neither side announces again what the other already has. Interoperates with zit both ways. check-in: f704c6d58b user: [email protected] tags: feat/horizon-cones, trunk

A horizon pull announced the client's whole holding with igot on every request, and the origin answered with its whole kept inventory: one card per artifact, both ways, on every pull. draft-fossil-horizon-00 §6.2.6 replaces both with a position.

The origin values pragma horizon-cursor right after horizon-served in every reply to a horizon pull, as zit places it. Its position is rid order (the clone SEQNO sense), stopped short of the lowest phantom: a filled phantom keeps its old rid, so a cursor covering it would hide the arrival. Receiving a cursor, any reply that pulls announces nothing at or below it. A filled phantom that a check-in from an earlier receipt already named as parent is recorded as the reclassification position; a cursor below it is answered as if absent. A cursor beyond the origin's own position is ignored.

The client keeps, per origin and cone, the last reply's cursor and the rid count the origin holds through, sends the cursor on every request, announces nothing on a pull-only request and only what it appended on one that sends. Sessions name their origin with SyncOpts.Origin; without it nothing is kept and every request announces in full. check-in: b61bd3475c user: [email protected] tags: feat/horizon-cones, trunk

TestZitHorizonInterop took whatever zit was on PATH. A zit release from trunk reports the same version as the horizon-rfc build but has no --horizon flag, so the test failed in zit's argument parser. The new zitHorizonBinary probes the binary for horizon support, as zitBinary already probes for send-submanifests, and skips it, or fails the run under REQUIRE_ZIT_BIN=1, with the reason. check-in: 27b85364f2 user: [email protected] tags: feat/horizon-cones, trunk

The public horizon API in the shape approved: a horizon is part of Cone (NewHorizonCone, Cone.Horizon, Cone.WithHorizon), store cones carry it through SetStoreCone and SetStoreConeContext and the new CloneOpts.StoreCone, checkout cones refuse it with ErrHorizonCheckout, and the CLI takes --horizon and --leaves on repo clone and repo cone store and --no-horizon on repo cone store. check-in: 964b91f682 user: [email protected] tags: feat/horizon-cones, trunk

A store cone could carry a horizon only through internal/cone, so library users and the CLI had no way to clone into or edit a content-horizon store. Add Horizon, HorizonRule, NewHorizonCone, Cone.Horizon and Cone.WithHorizon over the internal cone (always the content cut), carry it through SetStoreCone/SetStoreConeContext (deepening fetches what the older check-in keeps) and CloneOpts.StoreCone, and add --horizon, --leaves, --no-horizon and --cone to repo clone and repo cone store.

A horizon is a property of the store, not of a checkout (draft-fossil-horizon-00 3.2), so checkout cones carrying one are refused with ErrHorizonCheckout, and widening the store from a checkout cone keeps the store's horizon. check-in: e5904fc85b user: [email protected] tags: feat/horizon-cones, trunk

Appendix A step 3 without the cursor: the client sends pragma horizon, the origin computes the held set over its dense graph, sends only the blobs above the horizon, inflates crossing deltas, sends no cluster, answers horizon-served content and refuses what it cannot cover (§6.4). Against a horizon-ignorant origin the client requests what its clusters name, as transit (§4.5 after zit#266). Interoperates with zit both ways. check-in: 84be5ad824 user: [email protected] tags: feat/horizon-cones, trunk

A structural-horizon clone lacks C's ancestry, so tag propagation has nowhere to start: every check-in in K would derive no branch, and closed markers and names set below C would vanish. The seed of draft-fossil-horizon-00 Section 5.2 restates the dense tag state at each entry of K so the unmodified derivation comes out exact.

internal/horizon/seed mints it at an origin (T cards per row in force at C and each check-in of K whose primary parent is outside K; D the latest summarized event rounded up to a whole second; U the origin's login), and holds it at a clone in a table of its own, outside the artifact store, so nothing that enumerates or serves artifacts can send it. It is applied when held, by Rebuild, and by seed.Apply, which the sync layer calls after each round. Replacing or discarding a seed re-derives the store, since only a re-derivation separates the old seed's rows from those real artifacts wrote. A Verifier hook carries the Section 5.2.4 provenance policy; no signing is implemented.

The crosslink-table clearing moves from verify into manifest.Rederive so Rebuild and the seed share one owner of re-derivation. The cards match zit's seed byte for byte in T and U, and each implementation derives from the other's seed what it derives from its own. check-in: 45fb2c7ed4 user: [email protected] tags: feat/horizon-cones, trunk

The Appendix A step 4 measurement: an opt-in corpus test reporting what a structural horizon saves over a content horizon at several depths. check-in: 5430186930 user: [email protected] tags: feat/horizon-cones, trunk

Opt-in corpus test (LIBFOSSIL_CORPUS) for draft-fossil-horizon-00 Appendix A step 4: for C at 10, 100 and 1000 check-ins back along trunk's primary-parent chain and at the check-in nearest a year before the tip, compare a content horizon's transfer with a structural horizon's under both held-set rules. It works on a copy of the repository. The seed is estimated as one control artifact with one T card per tag in force at each entry of K, uncompressed.

fossil-scm.org clone: 67894 artifacts, 37.2 MiB stored, 8425.2 MiB expanded (21008 manifests, 8.9 MiB stored, 1099.8 MiB expanded; no sub-manifest nodes; 6263 other structure, 691 clusters). Runtime 58s, peak heap 161 MiB. Share = (content - structural) / content.

descendants rule K below (n / MiB exp) content MiB exp structural MiB exp share exp share stored 10 back 12 20996 / 1098.9 1136.5 37.6 96.7% 49.0% 100 back 121 20887 / 1089.9 1262.0 172.1 86.4% 48.5% 1000 back 1746 19262 / 962.2 2341.0 1379.0 41.1% 40.0% 1 year (426 back) 796 20212 / 1036.1 1846.3 810.3 56.1% 44.5%

leaves rule: K = 20907 of 21008 at every depth; share 0.0% expanded, -7.7% stored (the seed outweighs the 101 manifests below). check-in: c6ef4af29f user: [email protected] tags: feat/horizon-cones, trunk

The re-pinned §4.5 (zit#266) splits the cluster rule by origin. An origin that says horizon-served announces what it keeps directly, so a horizon store must not request a name only a cluster gives it. A horizon-ignorant origin, fossil(1) among them, folds 100 or more unclustered artifacts into a cluster as it answers a pull and announces only the cluster, so a new check-in may be named nowhere else: the store must request what the cluster names, as transit, and classify what arrives. A horizon store acted on no cluster from either, and so never received such a check-in from fossil.

CrosslinkCluster now mints member phantoms in every store, as fossil does; the received-file filter already drops the cluster and the blobs below the horizon at the end of the transfer. Whether to act on a cluster is decided where the origin is known: once a reply of the session says horizon-served content, the sync client asks for no cluster catalog and the sync and clone clients store no cluster file card, so no cluster-only phantom is minted from such an origin.

A real fossil(1) case pulls a leaf announced only through the cluster fossil makes, and holds it and its blob without holding the cluster. check-in: 588d5b264e user: [email protected] tags: feat/horizon-cones, trunk

A horizon store asked every peer for the whole history and threw away what fell below C on arrival, so a horizon clone cost a dense clone's bandwidth. draft-fossil-horizon-00 §6 lets the request carry the horizon and the origin restrict what it sends.

The client sends `pragma horizon HASH RULE` after its path-cone pragmas on every clone and pull, never horizon-structural, and counts the replies that say `pragma horizon-served content` (HorizonServed on the clone and sync results, internal and public). It still classifies every reply locally, so a horizon-ignorant peer stays correct (§6.3).

The handler reads the pragma into the request's cone and serves it through the existing keep machinery rather than a parallel path: the pull inventory's keep set drops the blobs below the horizon (via ConeReferences.HorizonBelow) and every cluster; the clone walk walks the tree of a held-set check-in only, counts a parent as sent only inside the held set, and sends every sub-manifest node at its row while sending no cluster. A crossing delta is inflated through the same sourceKept test the path cone uses, charged to the round's batch. req-clusters under a horizon answers nothing, and every reply says horizon-served content.

A horizon store refuses, through refuseClone, a dense clone and a horizon whose held set is not inside its own, both computed over the store with horizon.Place. That is the spec's subset test; zit refuses any other C or rule. A path-only store now also refuses a horizon-only request, which keeps every path.

Interop with zit at horizon-rfc bbd1435674 holds both ways: zit clones and pulls at a horizon from the handler, the client from `zit serve`, and each side's horizon store refuses the other's dense clone. check-in: 352a0d9e0b user: [email protected] tags: feat/horizon-cones, trunk

zit#266 reported that a horizon clone pulling from stock Fossil never learns a check-in Fossil announced only inside a cluster: xfer.c clusters 100 or more unclustered artifacts and then announces only the cluster. §4.5 at 9bf7ebfe0f binds the cluster prohibition to an origin whose replies carry horizon-served, and has the clone request a horizon-ignorant origin's cluster members as transit and classify what arrives. HZ-4.5-1 is restated; implementing it waits for horizon-served, which the horizon pragmas bring. The sync draft on that branch differs from the trunk copy pinned here only by §9's editorial list of extension pragma names. check-in: d5926b325c user: [email protected] tags: feat/horizon-cones, trunk

A content-horizon store against any peer: below-horizon blobs it is sent are recorded as elided rather than stored, it holds and requests no cluster, and a below-horizon blob an operation needs is fetched on demand. check-in: 27b8439777 user: [email protected] tags: feat/horizon-cones, trunk

No peer speaks the horizon pragmas yet, so a content-horizon store receives whatever a dense peer sends (draft-fossil-horizon-00 §6.3). The received-content filter now applies the horizon's classification, read from internal/content rather than re-derived: blobs below the horizon become elided phantoms and are counted in the result's Elided. Blobs no held manifest names yet stay as unreferenced holdings, and retained delta bases stay transit (§4.2).

A horizon store also acts on no cluster (§4.5). Crosslinking a cluster applies its tag but makes no member phantoms. The filter drops the cluster's content, and classification elides any phantom that carries the cluster tag. The client stops sending req-clusters. Path-cone-only stores keep today's cluster behaviour.

Conformance cases run against go-libfossil's handler and fossil(1): clone and pull, deepen, demand fetch through UpdateContext, transit deltas, and cluster rules. check-in: f1f4240f40 user: [email protected] tags: feat/horizon-cones, trunk

Sparse-store classification sized to what it proves rather than to the store: file content is never read, completely linked check-ins answer from the crosslink tables, and a content horizon reads only its held set's trees. A path-cone store of fossil-scm.org classifies in about 2 s at a 65 MiB peak, where v0.14.0 failed with a byte-budget error. check-in: cd1b25e114 user: [email protected] tags: feat/horizon-cones, trunk

Sparse classification read, inflated and delta-expanded every held blob, file content included, under a whole-store byte budget. On a clone of the Fossil SCM repository (67,894 blobs, 8.4 GB expanded) LoadAbsences failed at once with "byte budget exceeded", so no path-cone or content-horizon store of a real repository could classify its phantoms, and pull, verify and compaction failed with it.

Reference facts now come from the tables wherever they provably stand in for raw cards. A blob the crosslinker recorded as no artifact and mlink names as a file version is never read. A completely linked check-in (event row, no orphan row, a primary parent edge) whose primary parent and baseline trees are themselves reproduced contributes its mlink rows as F cards and its plink, cherrypick and tagxref rows as references; by induction over the primary chain those rows give the same (name, file) pairs as its raw cards. Everything else is still read raw: unlinked and orphaned check-ins, roots, check-ins whose parent never arrived, other artifacts, and every check-in once the store has seen a sub-manifest, since trees are not crosslinked. Budgets are charged per round for the selected blobs and their delta ancestors only, and a base's bytes are released once its last delta is applied.

A content horizon now reads only its held set's manifests and the baselines and nodes they reach, on demand, with one visited set and a byte budget scaled to the held set.

On the Fossil SCM clone, cone {src} classifies in about 2 s at a 65 MiB peak heap, reading 8,655 of 67,894 blobs; the inside/outside sets equal the all-raw result, and Outside/Required agree on every artifact. check-in: b28c06506b user: [email protected] tags: feat/horizon-cones, trunk

The content horizon's store classification: a blob is elided when no check-in in the held set names it through its full tree, a kept blob is in both keep sets under a path cone, structure is never elided under a horizon, and a horizon the store cannot place elides nothing, as zit does. check-in: bc54a61b2c user: [email protected] tags: feat/horizon-cones, trunk

# Conflicts: # conformance/horizon_test.go # conformance/requirements/horizon.tsv check-in: 80c8df6a74 user: [email protected] tags: feat/horizon-cones, trunk

A store cone may now carry a content horizon (draft-fossil-horizon-00 §3.2, §4.1-§4.3). Classification computes the held set K from the store's own check-in graph and walks K's full trees -- F cards directly, through B baselines and through S sub-manifest nodes, with one visited set over structural names -- because mlink records a file version only where it changed, and a blob introduced below C and unchanged through C is named by C's tree alone. Nothing about K is stored: every classification recomputes it, so moving C or losing the cone is a reload, never a migration.

Blobs no K check-in names are elided; with path members as well a blob is kept only when it is in both keep sets. Under a horizon nothing structural is elided, by the horizon or by path members, and delta bases of held artifacts and shunned names keep their classes. A C the store does not hold elides nothing, as zit's classification does: eliding everything would leave the pull that brings a deepened C nothing to ask for. A structural horizon is refused by SetStoreCone until it is implemented.

internal/content now imports internal/horizon, so the held-set tests, which commit through internal/manifest, move to an external test package to avoid an import cycle. check-in: 2ea6479ba9 user: [email protected] tags: feat/horizon-cones, trunk

zit's author called horizon-rfc bbd1435674 ready to implement against, and zit#265 settled the leaves rule there against fossil's leaf: a childless check-in is one no P card names, open or closed, and K under the leaves rule is C's descendants plus every descendant of each check-in where a childless check-in's ancestry meets C's. Both rules now yield a set closed under descendants, so leaves and branch tips derive exactly.

HeldSet follows zit's computation at that check-in (src/sparse.zig, ClassifyState.extend): meeting points are the members of C's ancestry a childless check-in's parent walk reaches without passing one, and a childless check-in is skipped only when it descends from C. The tests that encoded the old rule are restated: a merged-but-open branch tip adds nothing, a closed branch's tip does, the fork's descendants on C's side are held, and an unrelated root adds nothing.

The draft copy, the README pin and the table follow: HZ-3.1-1, HZ-5.2-1 (the seed applies to every entry of K) and HZ-6.2-11 (announce only in reply to a request that pulls) are restated, and HZ-3.1-4 (closure under descendants) is new, with a case checking that no child of a held check-in falls outside K under either rule. check-in: a68f89e414 user: [email protected] tags: feat/horizon-cones, trunk

Both slices defined the descendants/leaves rule: cone.HorizonRule for the serialized member and horizon.Rule for the walk. One concept gets one type; the walk now takes the value the cone carries. check-in: 6d8ccec5da user: [email protected] tags: feat/horizon-cones, trunk

The held set K of draft-fossil-horizon-00 §3.1 under both rules, bounded and computed from the check-in graph the repository holds. Conflicts in the horizon suite and table resolved by keeping both slices' cases and rows. check-in: 7a5ae7cee7 user: [email protected] tags: feat/horizon-cones, trunk

draft-fossil-horizon-00 Section 3.1 defines K as a function of C, the rule and the check-in graph, recomputed by whoever needs it and stored by no one. internal/horizon.HeldSet computes it from the plink and leaf tables: one bounded plink load, then iterative breadth-first walks.

Merge parents are followed both ways, as zit's walk does (src/sparse.zig, ClassifyState.extend and walkParents). Under the leaves rule each open leaf's ancestry is held up to where it meets C's ancestry, inclusive, so every equally near common ancestor the walk reaches is held. "Open leaf" is fossil's leaf table minus active closed tags rather than zit's "no held child", which keeps a below-C branch merged into a descendant held back to its fork, so any two open leaves keep their merge base in K. C that is not a held check-in is ErrNotHeld; phantom parents are never entered.

Section 8 requires the walk to carry visited-node bounds: the load and every walk share the 1<<20 bound of the other graph walks and fail with ErrBound, and cancellation is honoured. The conformance suite gains cases for HZ-3.1-1 and HZ-3.1-3, both PARTIAL until the horizon member and the content horizon exist. check-in: 75ded1b0e0 user: [email protected] tags: feat/horizon-cones, trunk

The horizon member of a cone and its version-3 serialization, byte-identical to zit's (draft-fossil-horizon-00 §3.3). check-in: 6da46813f9 user: [email protected] tags: feat/horizon-cones, trunk

A horizon names one check-in by full hash, a held-set rule and a depth. Version 3 carries it between the count and the flagged members, a distinct version so a reader predating horizons refuses the cone instead of reading the lead-in as path members. Forms without a horizon are unchanged, so their digests hold. A horizon-only cone has no path members and is path-dense. Parse re-serializes and compares, so an upper-case hash or any other non-canonical input is refused.

HZ-3.3-1 and HZ-3.3-2 are now covered. check-in: 5f637a3b62 user: [email protected] tags: feat/horizon-cones, trunk

Moves the horizon copy from 86d4b67372 to e6ee276c89, the check-in the sparse-cone copy is already pinned at. Every change between the two is in §6: §6.1 lets any request carry pragma horizon-cursor, §6.2 adds the SHOULDs to keep the classification and the structural graph per store prefix, and §6.2.6 defines the cursor. Five new rows read them into the table (HZ-6.2-9 to HZ-6.2-13) and HZ-6.1-1 names the cursor; the table is now 50 requirements, none implemented. check-in: be103fe35a user: [email protected] tags: feat/horizon-cones, trunk

chore(release): v0.14.0 changelog leaf check-in: 6d5357235a user: [email protected] tags: main, trunk, v0.14.0

feat: sub-manifest trees and sparse cones (draft-fossil-submanifests-00, draft-fossil-sparse-cone-00) check-in: 7244496a09 user: [email protected] tags: main, trunk

StartServer kills the server's process group and reaps the leader, but fossil server forks a child per connection, and SIGKILL reaches that child asynchronously. On a loaded machine the child could still finish a write after Wait returned, and t.TempDir's RemoveAll failed with "directory not empty" (TestCloneFromRealFossilUncompressedReply, load average ~20). This is the race #352 narrowed; cleanup now also waits, for at most five seconds, until the group has no member left. leaf check-in: f1b6155c8f user: [email protected] tags: feat/submanifest-reader, main, trunk

Five fixes the three-tool round trip and the fossil-scm.org comparison surfaced: an uncovered clone is refused with the push card first, repo status reports the checkout's changes, repo clone remembers last-sync-url, repo verify reports damage, transit phantoms and elision, and adding a directory adds the files beneath it. check-in: b3c5ad7bca user: [email protected] tags: feat/submanifest-reader, main, trunk

Manage stored a named directory, "." included, as a tracked file, and the next commit failed reading it. A named directory now expands through the checkout's bounded, cone-aware walk to the files beneath it, skipping paths with a dot component as fossil's add does without --dotfiles; a file named explicitly is kept as given. An uncovered path is refused as such before its absence on disk is reported. check-in: 9e28f473cc user: [email protected] tags: feat/submanifest-reader, main, trunk

repo verify hashed the held blobs and printed a fixed line. It now runs the full verifier the library uses for rebuild, prints each issue, and reports transit phantoms apart from elided artifacts, as sparse-cone 5.2 requires of a verification operation: a store complete within its cone verifies healthy and shows its elided count. check-in: 168062ca33 user: [email protected] tags: feat/submanifest-reader, main, trunk

go-libfossil read last-sync-url as the default peer for repo cone store, widening and required-content recovery, but nothing wrote it, so only a repository fossil had cloned had one. repo clone now stores its source as fossil clone does, with any user name and without the password. check-in: caf59be3b2 user: [email protected] tags: feat/submanifest-reader, main, trunk

Run from inside a checkout with the default --dir ".", repo status reported every tracked file MISSING: filepath.WalkDir's root entry is named ".", and the hidden-directory rule skipped it, and with it the whole tree. The command also compared the disk with tip rather than the checked-out version, so a checkout of an older version read as edited.

The command now opens the checkout and prints Checkout.Status, which classifies changes against the checkout's version as fossil's changes command does, cone included, with fossil's labels. Untracked files are no longer listed; fossil status leaves those to fossil extras. check-in: ea23bc8a41 user: [email protected] tags: feat/submanifest-reader, main, trunk

The safety checks ran while simio's Buggify was armed, so the read-path sites in blob decompression and content expansion truncated or flipped the bytes each check expanded. Every seed at the adversarial and hostile levels then reported a blob-integrity violation that was never persisted; CI runs the normal level only, so nothing caught it.

simio.SuspendBuggify switches injection off and returns a resume that restores the previous state with its random stream intact, so a check mid-run does not change the run's later choices. Every exported check in dst/invariants.go suspends for its duration. A guard test arms Buggify over a clean repository and fails without the change.

Also records in the CHANGELOG that ExtractOpts and RepoOpenCmd are no longer comparable. check-in: dfdf391349 user: [email protected] tags: feat/submanifest-reader, main, trunk

zit #262 settled the pull-inventory cost in the draft: a dense inventory can cost a requester the out-of-cone blobs of check-ins announced in the same inventory, since those names stay transit phantoms until a held manifest classifies them, and the filtering of 6.3 elides them on arrival. That is what the requester here does, so SC-6.2-4 closes. Its case now checks both halves: a blob announced with its check-in is never held, and one announced after it is never requested. check-in: 07d79d60eb user: [email protected] tags: feat/submanifest-reader, main, trunk

The draft now settles zit #257 to #260, so the four rows that waited on it close against code:

- Compaction and local filtering keep every manifest and sub-manifest node, interiors included (5.4, 6.3). RetainProvenance previously let an outside interior whose children were all held leaves go; the draft now requires it held, since it is the range proof that reaches the leaf. - A server answering a cone request acknowledges the reply with pragma cone-served in its leading pragma run (6.3). This server applies a request's whole cone or refuses it, and a refusal resets the reply, so the pragma is never sent for a refused or dense reply. SyncResult and CloneResult report ConeServed beside Elided, now public too. - An empty cone is refused (3.2); a new row and case pin it, locally and over the wire. - The per-round bound covers the tree walk only (6.2). A new row tests that a requester does not act on a dense inventory. It stays partial: an out-of-cone blob announced in the same reply as its check-in cannot be classified yet and is fetched, then filtered; zit #262 asks the draft to account for that cost. check-in: b5a64230b3 user: [email protected] tags: feat/submanifest-reader, main, trunk

A sparse clone sends the spine, in-cone nodes and in-cone blobs of draft 6.2; a held blob no F card names is none of these, so it is not sent, though zit's keep set holds it. check-in: 4529a3f71c user: [email protected] tags: feat/submanifest-reader, main, trunk

The per-round walk serves a cone clone, and nothing but tests reached the prepared index or the service that built it, so both go with their successor index. The whole-store reference loader stays for the pull inventory, compaction, verification and crosslink, and keeps its budget tests. The shun and private gates the prepared tests covered are now checked against the walk. check-in: ef6efa9f1e user: [email protected] tags: feat/submanifest-reader, main, trunk

A sparse walk now sends one clone-cap cookie per round naming the row the next round starts at, with a part-walked check-in's progress. A request that returns to that check-in without echoing its progress comes from a client that drops it; the round steps past the row instead of resending the same batch forever, and the next pull fetches what the walk owed by name. A gimme in a cone request is answered densely, since the requester names it (6.4). The SC-6.2-1 note records the pull inventory's whole-store read, which matches zit's. check-in: 6766dd6cba user: [email protected] tags: feat/submanifest-reader, main, trunk

Revert wrote a restored file with vfile's bits, which a pending merge may have changed, and left them in vfile. Fossil's revert writes the file with the reverted-to version's permission and records it in vfile.isexe and islink, so reverting a file whose only change is a merged executable bit takes the bit back off. versionPaths now carries each file's permission. check-in: da5d6546bb user: [email protected] tags: feat/submanifest-reader, main, trunk

Apply now does what fossil's stash_apply does when a stashed file changed since the stash: it merges the stashed change (pivot: the stash baseline) onto the file on disk with merge_3way, writing fossil's conflict markers and NAME-baseline/-original/-merge on conflict, an empty file for binary content, and reporting each conflict. Previously the stashed bytes were restored verbatim, losing the current change.

Save reverts a file with a pending merge to the checked-out version's executable and symlink bits, and records them in vfile, as fossil's revert does; the stash keeps the merged bits and the pop restores them. checkout.VersionPaths becomes VersionPerms to supply those bits.

The sparse stash-record-only conformance case now expects fossil's result: a file the merge took unchanged keeps the checked-out content after a stash round trip. check-in: 04e3ea8301 user: [email protected] tags: feat/submanifest-reader, main, trunk

Committing deltifies each manifest against its successor, so the first check-in's chain puts later manifests on the wire before their rows. A sparse round skipped those rows as already sent and never walked them, leaving their in-cone files to a later pull. The row is now walked without being sent again. Walk-only clones in the new cases hold no transit phantoms. check-in: 6791486470 user: [email protected] tags: feat/submanifest-reader, main, trunk

A clone round now reads and walks only its batch. What remains of the MUST is a cone-scoped pull, which still classifies the whole held store to filter its igot inventory and gimme answers. The walk also carries its request context explicitly. check-in: cbc0f9a21e user: [email protected] tags: feat/submanifest-reader, main, trunk

A cone-scoped clone no longer classifies the whole held store before its first round. Each round reads only the rows it reaches: structural artifacts go out at their row, and a check-in's row walks its tree, descending only nodes whose key range meets the cone and that its primary parent's tree does not carry over the same range, then sends the in-cone nodes and blobs that walk reaches. Content and sub-manifest rows wait for those walks and are charged as excluded rows are, so a round's work tracks its batch. A check-in whose walk outgrows a round resumes in the next one from progress kept in the clone-cap cookie. Delta bases the round did not reach are served in full form. The walk keeps the depth and visited-node bounds per request.

A round spent wholly inside an excluded run sends nothing yet advances its cursor; a coned clone now continues on that progress instead of reporting that the server stopped sending.

HandleEnd reports the artifacts a cone request classified. check-in: 2909f2a676 user: [email protected] tags: feat/submanifest-reader, main, trunk

Save no longer refuses while a merge is pending. As fossil's stash save, it stashes each changed file against the content last written for it (mrid) and reverts only the stashed files, so vmerge is kept and the next commit records the merge parent. A file the merge added is not in the checked-out version, so its revert deletes it.

Apply now fails before writing anything when a stashed modified file is missing from disk, as fossil's pop fails with "no such file" and rolls back; this is what a merge-added file meets after the save.

In a sparse checkout the record-only merge outside the cone is left pending and unmaterialized, and the in-cone files round-trip. check-in: c314e928f8 user: [email protected] tags: feat/submanifest-reader, main, trunk

In the linked crash state both marker names are one file, and SQLite on Linux warns about the multiple links on stderr while fossil lists the record normally. CombinedOutput folded the warning into the file list. check-in: 83f574dc8f user: [email protected] tags: feat/submanifest-reader, main, trunk

Reverting a merge's out-of-cone changes by name, and with no names, leaves the record exactly where fossil's revert leaves the dense analog, without materializing anything. On a sparse checkout stash refuses a pending merge with the dense checkout's refusal, round-trips an in-cone edit as fossil does, and refuses to apply an out-of-cone file, naming the cone and keeping the entry. Clears the SC-4.4-1 scope note. check-in: 902108ace9 user: [email protected] tags: feat/submanifest-reader, main, trunk

A crash between moveMarker's link and unlink left one checkout database under both the native and the private name, which discovery refused, so the checkout could not be opened again. moveMarker now records its target name in the database before linking, and Open completes the move to that name. A still-running transition keeps the same name, so an Open racing it converges instead of removing both names, and moveMarker treats an old name already removed as done.

Crash injection at every step of narrowing, widening and legacy migration reopens as the old or new state with files matching the cone; stock fossil either refuses the directory or lists the dense record, before and after recovery. Clears the SC-4.1-2 scope note. check-in: bb9805a794 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # cli/repo_cone.go # conformance/requirements/sparse-cone.tsv # conformance/sparsesync_test.go check-in: afd72285e5 user: [email protected] tags: feat/submanifest-reader, main, trunk

Section 5.1 permits holding out-of-cone nodes while 5.4 and 6.3 read as dropping or eliding them; zit holds them and so do we. The notes keep the rows PARTIAL until the draft says which sentence governs. check-in: 8ea7bd1ec4 user: [email protected] tags: feat/submanifest-reader, main, trunk

(cherry picked from commit 1e7b5f246ef9f5af5b366140eb03e7a80227f9bb) check-in: 42a5ef6dd7 user: [email protected] tags: feat/submanifest-reader, main, trunk

Repo.SetStoreConeContext replaces the store cone and pulls newly covered elided artifacts from an explicit peer through ordinary cone-scoped sync; with no peer it refuses naming the elided count, and a cancelled context refuses before any request, both leaving policy unchanged. repo cone store shows or replaces the store cone, defaulting the peer to last-sync-url. Closes SC-6.4-1 and SC-6.4-2. check-in: 08efefff20 user: [email protected] tags: feat/submanifest-reader, main, trunk

An aware clone and pull hold exactly the spines, in-cone nodes and in-cone blobs and never request or receive the remainder. Against a cone-ignorant server the requester elides exactly the blobs every F card names out of the cone and keeps every structural artifact, as zit does. Expected sets come from the origin's decks. check-in: af86aff844 user: [email protected] tags: feat/submanifest-reader, main, trunk

Extract now runs fossil's repo-cksum check after materializing: a present R card is compared with the aggregate over every file the manifest names, read from the store, so a sparse checkout over a dense store reaches the same verdict as a dense one, and a sparse store skips the check. Like fossil, a disagreement is a warning delivered through ExtractOpts.Warn and never fails the extract; repo-cksum off disables it. The conformance case builds its repository with fossil and checks fossil warns exactly where go-libfossil does. Clears the SC-4.3-4 scope note. check-in: 318f46de9e user: [email protected] tags: feat/submanifest-reader, main, trunk

A sparse store refuses a clone its cone does not cover unless the boolean partial-mirror repository setting is on, read as fossil reads a boolean setting. It reaches HandleSync, XferHandler, ServeHTTP and repo serve alike; libfossil.SettingPartialMirror names it. Closes SC-5.3-2. check-in: 6012d82bd6 user: [email protected] tags: feat/submanifest-reader, main, trunk

The public entry point refuses a dense store, as zit's pack --drop-elided does, and drops exactly the blobs that only out-of-cone F cards name. The test computes that set from the held artifacts, then verifies the store and recovers every dropped payload by widening through SetConeContext. check-in: 774f579bfa user: [email protected] tags: feat/submanifest-reader, main, trunk

A cone-excluded row is charged at its expanded size plus card overhead, so a round over a long excluded run stops where a dense batch would and the cursor resumes it, instead of walking the whole run in one round. SC-6.2-1 stays partial: ordinary keep-set discovery still reads the whole store. check-in: 1da7f0f002 user: [email protected] tags: feat/submanifest-reader, main, trunk

Many check-ins sharing one wide sub-manifest cross the 1<<20 visit bound at depth two; 1023 roots are served and 1024 are refused the way depth 17 is. Clears the SC-8-2 scope note. check-in: 8652936469 user: [email protected] tags: feat/submanifest-reader, main, trunk

Widening a sparse-store checkout with a peer fetches and materializes exactly the newly covered payloads; with no peer it refuses naming the elided class and changes nothing; a peer lacking the content is refused with the unavailable artifact named. Clears the SC-3.5-2 scope note. check-in: fcb7f90f2c user: [email protected] tags: feat/submanifest-reader, main, trunk

The sparse-store entry said nothing yet fills a store sparsely and the sparse-checkout entry said a merge touching an uncovered file is refused; both are no longer true on this branch. check-in: 7720f39cd0 user: [email protected] tags: feat/submanifest-reader, main, trunk

The checkout lane's review blockers are fixed and re-reviewed, so the two scope notes now name only what is unverified: durable crash recovery of the private marker, a conformance case for reverting a record-only change, and stash apply on a sparse checkout. check-in: 7163d16293 user: [email protected] tags: feat/submanifest-reader, main, trunk

Only managed revert steps reach it, each with a resolved path, so the fallback for an empty path was dead code standing in for a broken invariant. check-in: a3cc82b191 user: [email protected] tags: feat/submanifest-reader, main, trunk

Record-only merge steps return before the content switch, and a both-changed path outside the cone is refused, so storeMergePayload had no production caller. Its tests called it directly; they go with it, and the merge-failure test keeps its vmerge assertion. check-in: dac73e55c9 user: [email protected] tags: feat/submanifest-reader, main, trunk

A partial commit wrote every pending change outside the cone into the check-in. Such changes now behave as disk changes do: a full commit records them, a partial commit leaves them pending against the new version, and a partial commit is still refused while a merge is pending. Fossil-oracle tests cover the dense analog of each case. check-in: 802906e3b8 user: [email protected] tags: feat/submanifest-reader, main, trunk

After the observed generation changes, every request falls back to dense and the service never re-prepares. Say so where the type is documented, so a caller does not expect sparse serving to resume after a foreign commit. check-in: 322c8ef406 user: [email protected] tags: feat/submanifest-reader, main, trunk

Preparation counted newlines in every artifact against the 1M card budget, so a repository holding one large multi-line text file was refused far below its byte limits. Only an artifact with a verified Z card can parse as a deck, so only those are charged cards and parsed; the byte budgets still bound all content. check-in: 2f0f64f464 user: [email protected] tags: feat/submanifest-reader, main, trunk

The prepared index left private and shunned RIDs out, so a gimme for an in-cone private artifact failed keeps() and got no reply, where the dense handler sends the private igot. A client never told the artifact is private can keep requesting it. The index now answers cone membership only, as the dense keep-set does; the handler's live private and shun gates decide visibility, so no payload is exposed. check-in: 87f44dbf99 user: [email protected] tags: feat/submanifest-reader, main, trunk

Fossil's update leaves vmerge alone, so the next commit records the merge parent. The sparse update cleared vmerge, losing the parent and letting a partial commit through while the merge was still pending. check-in: 5a070508db user: [email protected] tags: feat/submanifest-reader, main, trunk

The fallback that explains a parent whose tree the store does not hold ran under context.Background, so a cancelled commit still paid for it. Commit passes its context; snapshot, which has no deadline, says so explicitly. check-in: c25e714b08 user: [email protected] tags: feat/submanifest-reader, main, trunk

The closure walk parsed a node before checking whether its frame had been visited, so a tree whose S cards name one sub-manifest many times parsed it once per reference: a 616 KB node took 0.46s at 50 references and 3.7s at 400. The visit and byte limits bound what is held, not that work, and a caller without a deadline could wait for days inside the budget.

The walk is its own function now, parsing through a callback it calls at most once per (artifact, baseline) pair, so a test can count parses. check-in: 6824f7701a user: [email protected] tags: feat/submanifest-reader, main, trunk

(cherry picked from commit f10fb0eca15614a98bdecfc9d715464615738924) check-in: db1f83c19a user: [email protected] tags: feat/submanifest-reader, main, trunk

(cherry picked from commit fc1c110fa16c178c73a4bbf937c1f01cc41d5795) check-in: 20b51362a4 user: [email protected] tags: feat/submanifest-reader, main, trunk

(cherry picked from commit 63520cd9e0d6ed4f96be8b302dfc217ee57da82f) check-in: 9d336c0907 user: [email protected] tags: feat/submanifest-reader, main, trunk