Timeline

An origin that held a delta whose source it never received could not be cloned at all. walkCloneChain found the source unforwardable (a phantom) and asked buildCloneArtifact to send the delta expanded; expanding it failed, the handler errored, and every client stopped at clone sequence 1.

The origin now withholds a chain that ends at a phantom, as it withholds a row outside the clone's cone. Sending the stored delta naming its phantom source, as fossil's send_compressed_file does, is not enough: a fossil client then rejects the whole clone for its unresolved deltas. A later pull fetches the chain once the source arrives. check-in: 67bc700498 user: [email protected] tags: main, trunk

fix(sync): ask for phantoms in hash order check-in: d40046f706 user: [email protected] tags: main, trunk

buildGimmeCards ranged over the phantoms map, so each round's gimme cards came out in Go's randomised map order. When a round has more phantoms than mxPhantomReq allows, that also picked a random subset to ask for. A seeded simulation then could not replay the same wire, and the docs generator's engine.json (which records a pull's cards) changed between runs, failing the generated-docs check about one run in three.

Sort the phantom names before building the cards. Phantom sets are bounded by what the peer announced, so the sort is cheap next to the round trip. check-in: 82dae665dd user: [email protected] tags: main, trunk

build(docs): one deploy build, through scripts/docs-build.sh check-in: 5526734cfd user: [email protected] tags: main, trunk

The deploy ran two builds: the Cloudflare trigger's build command and wrangler.jsonc's own. The triggers now have none, so wrangler.jsonc is the one place the deploy build is defined, and it calls the same script developers run. check-in: 235cf4fd8d user: [email protected] tags: main, trunk

docs: rework the site and landing page, generate its facts from the code check-in: 4caaddcd19 user: [email protected] tags: main, trunk

The Build (tinygo) job builds examples/tinygo on every change, so its size (3.52 MB raw, 1.26 MB gzipped) is the one the site quotes; the README keeps the macOS measurements beside it. check-in: 313f64a89e user: [email protected] tags: main, trunk

feat(wasm): build the whole public API with TinyGo check-in: b3680c4757 user: [email protected] tags: main, trunk

go-libfossil compiled for js/wasm only with the standard toolchain, whose output is about 16 MB, too large to ship with a page or an edge function. Three standard-library features TinyGo lacks kept it from building there: http.ErrUseLastResponse and net.DNSError in the sync client, and os.SameFile in checkout-marker recovery.

Each now sits behind a tinygo build tag. Under TinyGo a sync redirect ends the request with an error instead of being followed, DNS failures are not classified as transient, and two names of one checkout marker are reported as ambiguous rather than paired; the standard build is unchanged.

examples/tinygo exercises the whole public API, sync and signing included. Built with TinyGo 0.43 and wasm-opt -Oz it is 3.83 MB raw and 1.40 MB gzipped, against 16.1 MB and 4.06 MB from the standard toolchain. A new CI job builds it with TinyGo on every change. check-in: 0650171d55 user: [email protected] tags: main, trunk

go-libfossil now builds with TinyGo, sync and signing included (examples/tinygo). The landing page reads the measured sizes from data/wasm.json instead of an outside project's engine-only build, and the install page says how to build it and what differs under TinyGo. check-in: 7377cb7c97 user: [email protected] tags: main, trunk

Runs sync-version.sh, gen-llms-txt.sh and Hugo, given the hugo binary as its first argument. It needs no Go toolchain: the inputs that do are committed and checked for freshness in CI. make docs-build uses it. check-in: 1ef14e7167 user: [email protected] tags: main, trunk

The home page renders its hero figure, conformance table, version and footer from the generated data and site params. Docs pages keep Hextra with Fraunces, Inter and JetBrains Mono and rust as the accent in both themes. Add the get-started pages (install, CLI quickstart, embed in Go, first sync), the conformance reference rendered from data, and point the nav, README and old links at the new layout. check-in: 41dfd5c149 user: [email protected] tags: main, trunk

tools/docsgen/engine records one check-in as an artifact, as its derived rows and as the cards of a pull, from a real engine run with fixed inputs. tools/docsgen/conformance turns TestCoverage's report into per-draft and per-requirement data. scripts/gen-cli-docs.sh writes one page per command group from the libfossil binary's --help. gen-llms-txt.sh and check-llms-txt.sh are ported from dagnats, and sync-version.sh now takes the highest release tag and writes the go.mod version too.

scripts/check-docs-generated.sh regenerates everything into a scratch dir and fails on any difference; make docs-check and the test-docs CI job run it. check-in: d7fd910ffc user: [email protected] tags: main, trunk

Move the flat docs pages into sections, keep each old URL as an alias, split sparse cones from horizons, and add concept pages for artifacts, repository state, sync, sub-manifests and signing plus a settings reference. Audit the moved pages against the code and fix drift. check-in: 93ad669695 user: [email protected] tags: main, trunk

feat(signing): sign check-ins and horizon seeds with gpg or ssh check-in: 4a439c6de6 user: [email protected] tags: main, trunk

go-libfossil signed nothing and verified nothing, so a structural-horizon clone could not tell a seed its origin minted from one substituted on disk (draft-fossil-horizon-00 §5.2.4, HZ-5.2-5 and HZ-5.2-6), and a repository with fossil's clearsign setting on stored its check-ins unsigned.

Signing follows fossil's contract, the same one zit follows. With clearsign on (and the legacy omitsign off), every check-in a commit or branch creation stores is clear-signed by pgp-command: unset, gpg --clearsign -o, run as COMMAND OUTFILE INFILE; a command named ssh* runs as COMMAND INFILE and its detached signature is wrapped in fossil's SSH envelope. A signer that fails, or returns text that does not strip back to the manifest, abandons the commit. NoSign and --nosign store one check-in unsigned.

An origin with clearsign on signs each seed it mints. It signs only after the request is authorized, outside the request's transaction, keeps one signature per seed and command so the seed's name is stable, and re-signs a kept signature its own horizon-seed-key no longer verifies. Keeping is best effort, so a read-only mirror still serves.

A clone given its origin's key (CloneOpts.SeedKey, repo clone --seed-key, kept as horizon-seed-key) verifies every seed on clone, pull and rebuild, and falls back to the content horizon from one unsigned or signed by another key. An SSH key is verified in process: namespace fossilscm, the configured key, no SHA-1 ssh-rsa, and exactly fossil's envelope. An OpenPGP fingerprint is verified by gpg, which must report one good signature by that key over text byte-equal to what the parser reads, with nothing after the signature. A verifier that cannot run keeps the held seed and fails the sync instead of discarding it.

fossil 2.28 and zit read the signed check-ins; a check-in zit signs with the same SSH key verifies here; zit clones a signing origin and holds its seed. check-in: 612f2e7acc user: [email protected] tags: main, trunk

chore(release): v0.15.0 check-in: a992dcb34f user: [email protected] tags: main, trunk, v0.15.0

feat: horizon cones (draft-fossil-horizon-00) check-in: ac0bf85d72 user: [email protected] tags: main, trunk

The previous change let a sparse store link a check-in before its files. A pull that deepens a structural horizon then linked the new base first and left its files unrequested phantoms, and the merge over it found no content (TestRepoMergeDeepensAcrossHorizon).

A pull waits on a check-in's files again, except those the store cone's path members exclude, which never arrive. Only a clone into a sparse store waits on none: the origin's walk sends every file the store keeps in the same session, and nothing reads the store before the clone ends. check-in: 35566881ec user: [email protected] tags: feat/horizon-cones, trunk

A sparse store published no event or plink rows for a check-in while a P or Q source of it was absent, unless a structural horizon held it below. A source can be absent for good: fossil-scm.org's repository lacks the parents of five check-ins, which a dense clone links against phantoms, so a cone or content clone of it failed repo verify on them.

SparseCheckinReady no longer requires an absent source's file set: the check-in's spine is published against the phantom, as a dense store publishes it, and its orphan row keeps mlink derivation waiting until the source arrives. check-in: 69f3048e67 user: [email protected] tags: feat/horizon-cones, trunk

A cone-aware clone charged each row it walked past at its expanded size, while a dense round charges the bytes it sends, a row's stored delta. A sparse round of fossil-scm.org's repository covered a hundredth of a dense round's rows and the clone ran out of rounds. SC-6.2-1 bounds the walk as a dense clone bounds its batches, so a walked-past row is now charged its stored size, read from the four-byte prefix of blob.content without inflating it. check-in: 835557d4b1 user: [email protected] tags: feat/horizon-cones, trunk

A cone-aware clone classifies each row it walks, and each served delta's base, by parsing the row's bytes. The handler has no content cache in production, so every row's whole delta chain was expanded and hashed again; on fossil-scm.org's repository that was the cone clone's first round, unfinished after 20 minutes.

The walk now reads, once per request, the rows the crosslink tables prove to be file content (named by mlink, claimed by no artifact table; a store that has seen a sub-manifest answers nothing) and classifies only the rest, through its own unchecked cache: classifying serves nothing, and the bytes a row is sent with are still checked apart.

A file version's base is the newer version, which a walk in rid order has not reached yet, so every version went out whole: a 9 MB text per version of sqlite3.c. A content base the walk has not reached is now forwardable when the store's classification of the cone keeps it (SC-6.2-2 bars only a base outside the keep set). check-in: 9a2938b626 user: [email protected] tags: feat/horizon-cones, trunk

A clone's receive linker tested a deferred check-in's every F card again each time one blob it waited on arrived, and each time a manifest it was a delta of arrived, and its waiter index (8192 entries) filled after a few check-ins of a large tree. A sparse store's out-of-cone files never arrive, so a cone or content clone of fossil-scm.org's repository spent its first round, past 20 minutes, in availability queries.

A sparse store now waits on no file: it holds what its cone keeps, and SparseCheckinReady judges its completeness. A waiter strikes each arrival from its wait set and is examined again only once the set is empty, and a check-in already waiting is not re-examined when a delta source arrives under it. check-in: 09b62091ec user: [email protected] tags: feat/horizon-cones, trunk

A check-in crosslinked before its cherrypick ('+') source arrived kept no cherrypick row, since insertCherrypicks skipped a source it could not find, and so never received the auxiliary mlink rows for that source: the late-arrival pass walked plink alone, which holds no cherrypick edges. Canonical skips a missing source the same way and relies on the rebuild its clone ends with; a clone here links as artifacts arrive and has no such rebuild. A clone of fossil-scm.org's repository was left with 139 cherrypick rows and 262 mlink rows fewer than its origin.

insertCherrypicks now reserves a missing source as a phantom, as plink does for a missing parent, and the late-arrival pass (linkPlinkChildren, now linkLateChildren) walks the source's cherrypick children with plink's. check-in: 28f343c11b user: [email protected] tags: feat/horizon-cones, trunk

forwardableSource reads a row, its shun and private state and the tree gate each time it is asked, and a clone round asks about the same rows from every chain walk and every sentEarlier search: a twelfth of a dense clone's CPU. A request reads one snapshot, so the verdicts are kept for its duration.

TestCorpusClone logs an origin error, which its client sees only as an HTTP status. check-in: b9ec3c086e user: [email protected] tags: feat/horizon-cones, trunk

ReceiveLinker.LinkStored made a fresh availability cache for every blob it linked, so each deferred check-in's retry walked every F card's delta chain again; after the receive fixes that walk was a sixth of a dense clone's CPU. Only the unavailable verdicts can go stale during a receive, which only adds content, and a clone whose round fails stops receiving, so LinkStored now drops just those (AvailabilityCache.ForgetUnavailable) and keeps the rest. check-in: a30155185d user: [email protected] tags: feat/horizon-cones, trunk

A clone round sends each row with its delta chain, source first, but only remembered what it sent within the round. fossil deltifies each version of a file against the next, so the rows of a file's versions come in rid order, each a delta on a later one: the oldest version's round sent the whole history, and each later version's row, reached in a later round, sent the rest of it again. Cloning the Fossil repository received 333,063 delta cards by round 64 for 22,353 new artifacts, each checked again by the client.

The origin now infers what earlier rounds sent: a row below the cursor that its round sent as itself reaches, through sources this clone forwards, every node of its chain. sentEarlier walks down the delta table from a row or a chain source, at most 256 rows, and a row it finds sent is skipped like one sent earlier in the round, and a chain stops at a source it finds sent. Not finding one sends again, which is only redundant. A row below the cursor filled or published after its round passed it is taken as sent; rows below the cursor are not sent again in any case. check-in: 07b30dccde user: [email protected] tags: feat/horizon-cones, trunk

The real-repository gate found a dense clone of the Fossil repository CPU-bound past an hour, in delta.Checksum and chain expansion. checkDelta rebuilt each received delta's basis from the root of its chain with content.ExpandUnchecked, after walking the chain for availability; a file's history arrives as one chain of hundreds or thousands of deltas, so the cost was quadratic in its depth, as Repo.Verify's was before 1a5abca.

A receive pass now rebuilds the basis through an unchecked content cache (content.NewUncheckedCache): from the deepest node of the chain it holds, keeping each node it rebuilds, and it remembers every artifact a card stored and verified once the card's savepoint commits. A clone keeps one such cache across its rounds and clears it when a round rolls back. A basis the cache holds, or one dangling in the pass, needs no availability walk, and the clone's check for a basis to request uses the same knowledge. The cache holds what each row rebuilds to, unchecked as checkDelta's basis always was, so every verdict is unchanged; a verified delta's target also goes to the receive linker, which no longer expands it again.

TestCorpusClone (LIBFOSSIL_CORPUS) clones a copy of a repository from an in-process server for profiling. check-in: b43fd24a1f user: [email protected] tags: feat/horizon-cones, trunk

Every request ran in a write transaction (BEGIN IMMEDIATE) for as long as it was served, a horizon request read the classification memo in another, and each request created the table-sync schema on a repository without it. A clone arriving while another request was served waited for the write lock past the 5s busy timeout and got HTTP 500 "database is locked (SQLITE_BUSY)"; its client gave up after four attempts. The real-repository gate hit it with a horizon clone beside a dense one.

A request none of whose cards can write now runs in a read transaction (db.WithBoundReadTx), which takes a snapshot without the write lock; the memo read does too; loading synced tables only reads, and a schema card creates the schema it registers into. Requests that write keep their write transaction. v0.14.0 has the same write transaction per request: a clone from its server fails the same way while another connection holds the lock. check-in: f8bb98e196 user: [email protected] tags: feat/horizon-cones, trunk

An origin answering a horizon request never sends a cluster, even one a gimme names; an origin may name a history URL in horizon-served (horizon-history-url), which the client reports; and a merge with no peer to ask refuses whenever either side's ancestry crosses the horizon. Conflicts in the changelog and the CLI horizon tests kept both sides. check-in: c4b191ed05 user: [email protected] tags: feat/horizon-cones, trunk

draft-fossil-horizon-00 §5.5 has a structural clone ask the origin for the base whenever either side's ancestry crosses the horizon, even when its own walk finds a common ancestor: over part of the graph that ancestor need not be the nearest. Without a peer Checkout.Merge and Repo.Merge refused only when the walk itself crossed, and merged over a held base whose sides' ancestry crossed below it.

Both now refuse such a merge with an error matching ErrBelowHorizon that names the horizon and points to Checkout.MergeContext with a Peer; MergeContext refuses the same way when the origin does not answer. A merge whose ancestry stays above the horizon, or one given its base, still merges locally. The CLI's merge already falls back to asking the remote on ErrBelowHorizon, so it now asks in these cases too. check-in: 263de5a736 user: [email protected] tags: feat/horizon-cones, trunk

The horizon draft pinned at zit horizon-rfc 5d618aaeb1: an origin with no tag in force mints no seed and serves the content level, and choosing C is shown with each rule's cost (repo cone cost, Repo.HorizonCost). check-in: 77e74bd011 user: [email protected] tags: feat/horizon-cones, trunk

Draft §3.4 asks a tool that suggests a horizon to show, before the choice, what each rule would hold in check-ins and bytes. repo cone cost <check-in> prints that for descendants and leaves from the local repository, and Repo.HorizonCost returns the figures. The --horizon help of repo clone and repo cone store points to it, noting that a clone has no repository yet. check-in: 834b28d9fe user: [email protected] tags: feat/horizon-cones, trunk

The draft now says an origin mints no seed, and serves the content level, when no tag is in force at any check-in the seed would cover, since a control artifact needs a T card. HZ-5.2-2 loses its scope note and the new HZ-5.2-7 row is proven by a case that mints none, advertises horizon-served content and leaves the clone holding no seed. HZ-3.4-1 is restated as guidance for the tool; it is covered with the cost report in the next commit. check-in: 4e9dea8cc1 user: [email protected] tags: feat/horizon-cones, trunk

Annotate in a structural store marks the lines that reach the horizon and names the deepen that would show their origin (§5.5); the origin memo keeps the trees a horizon's walk reads in a bounded cache, so a second horizon over the same store reads only trees the first did not. check-in: 479cbaad12 user: [email protected] tags: feat/horizon-cones, trunk

draft-fossil-horizon-00 §6.2 asks an origin to read each structural artifact's references once per store and classify every cone over that graph. The memo did so for kinds and P, B, S and A cards, but the F cards of a horizon's trees were walked from the store by the first classification of each horizon, so a second horizon under another C or rule re-read every tree it shared with the first.

The memo now keeps, per store lineage, each manifest, baseline and node a walk read: its own F-card targets and its B and S references, so a delta manifest shares its baseline's entry. A view stages what it reads and the memo takes it on Publish, so a request that rolls back leaves nothing; a kept piece serves a view only when its rid is held at that view's prefix, and its size is charged as a read would charge it, so budgets fail where a fresh walk fails. Names are interned and the cache is capped at 64MiB with LRU eviction: fossil-scm's history names about 21M F entries, over 300MiB uncapped.

On a copy of a 21k-check-in fossil repository, horizon B (2000 check-ins back) classified after horizon A (1000 back) took 2.5-3.0s and now 1.6s, reading 995 trees instead of 1995; A is unchanged at about 1.3-1.6s. check-in: 13f8021952 user: [email protected] tags: feat/horizon-cones, trunk

draft-fossil-horizon-00 §6.2.5 lets an origin append to pragma horizon-served the URL of an interface outside the protocol at which history below a horizon can be queried, and §5.5 has a client direct the user there. go-libfossil emitted the level alone and dropped any URL a reply carried.

An origin now appends its horizon-history-url setting when set. The value rides the pragma as one raw card token, so SetConfig refuses anything but a single http or https URL with a host and no white space or credentials, and the handler emits nothing for a value stored past that check. The client records a valid URL from any origin in its clone and sync results, exported additively on the public results, and the CLI's clone and cone store/deepen print it. check-in: fe06d02a12 user: [email protected] tags: feat/horizon-cones, trunk

draft-fossil-horizon-00 §5.5 requires an operation needing history below the horizon to detect the crossing and name the horizon. Annotating back past C stopped at the parent the store does not hold and credited every remaining line to the oldest version it reached, presenting a guess as an attribution.

The walk now asks content.BelowHorizon about the parent it could not load; when it lies below the structural horizon, the lines left without credit are marked BelowHorizon and reported against the horizon's check-in. The public AnnotatedLine carries the flag, and repo annotate prints such lines against ^<horizon> with one trailing note naming the horizon and repo cone deepen. A content horizon holds every manifest, so BelowHorizon never fires there; a conformance case clones both depths to show it. HZ-5.5-1 is now covered for every operation it lists. check-in: 1823fe8ca8 user: [email protected] tags: feat/horizon-cones, trunk

An origin answering a horizon-scoped request already announced no cluster in its clone, pull or sync inventory and answered req-clusters with nothing, but a gimme naming a cluster still got the cluster's file card. A horizon clone that learned a cluster's name anywhere could so still be sent one, against draft-fossil-horizon-00 §4.5 and §6.2.

The handler now drops a cluster from a gimme's answer when the request carries a horizon pragma. A new conformance case records the origin's replies to a horizon clone, pull and sync across freshly minted clusters, with and without a cursor, and to every gimme and req-clusters shape, and checks no cluster name crosses; an interop test checks zit's origin the same way. check-in: 5768a5342d user: [email protected] tags: feat/horizon-cones, trunk

The horizon draft re-pinned to zit horizon-rfc 24457c7dd6, the batch agreed with zit's author: the list-form merge-base query (at most two secondaries; silence when a named check-in is unknown), must-ask when either ancestry crosses the horizon, deepen refusing without a common ancestor, boundary labels over every held manifest, the seed's date and label, the structural fallback dropping the seed, and cursors voided when the leaves rule's held set grows below them. check-in: c216f38b6a user: [email protected] tags: feat/horizon-cones, trunk

Record what the re-pin changed in behavior: the list-form merge-base query and when a merge asks it, the seed's U label, the deepen refusal's remedy, and the cursor voided by a leaves-rule fork. check-in: bb5fc8ebff user: [email protected] tags: feat/horizon-cones, trunk

The re-pinned draft makes the parents of a held baseline below C boundary hashes (5.3, 6.2.2) and states the structural store's fallback for a pull or sync (6.2.5, 6.3). go-libfossil already met both; these cases check them against the final text.

A case on the delta-baseline fixture (c3 and c4 delta manifests on c2, C = c3) checks that c1, the baseline's parent, is labelled no later than the reply delivering c2 and is held as an elided phantom the clone never asks for. An interop test runs the fallback with zit both ways: zit's structural store pulls from go-libfossil serving as a horizon-ignorant peer, and go-libfossil's pulls from a zit that cannot place C; each falls back to the content horizon, discards its seed and fetches the manifests below C. The recording origin gains an ignorant mode for it. check-in: a553cc1e11 user: [email protected] tags: feat/horizon-cones, trunk

Under the leaves rule a new childless check-in forked below C can move the meeting point of its ancestry with C's deeper, and older check-ins join K together with their blobs. Those sit below every cursor the origin had handed out, so a client holding one was never told of them: the origin recorded a reclassification only when a phantom was filled (6.2.6 requires recording any change of the classification).

The origin now also voids a cursor when K holds a check-in at or below it that the held set computed over the check-ins it held through that cursor lacked (horizon.Graph.JoinedAfter, a walk bounded by rid). Like the slot walk's void it is a function of the store, so the memo and a fresh classification agree: TestHorizonMemoEqualsFresh compares them at every cursor and now requires some world to void this way, and a cursor conformance case pulls under the leaves rule across such a fork, the stale cursor answered as if absent once and the next one honoured. check-in: 2c0329ede4 user: [email protected] tags: feat/horizon-cones, trunk