Timeline

fix(sync): clone an origin that holds a dangling delta check-in: 77c6eb043e user: [email protected] tags: main, trunk

An origin that held a delta whose source it never received could not be cloned at all. walkCloneChain found the source unforwardable (a phantom) and asked buildCloneArtifact to send the delta expanded; expanding it failed, the handler errored, and every client stopped at clone sequence 1.

The origin now withholds a chain that ends at a phantom, as it withholds a row outside the clone's cone. Sending the stored delta naming its phantom source, as fossil's send_compressed_file does, is not enough: a fossil client then rejects the whole clone for its unresolved deltas. A later pull fetches the chain once the source arrives. check-in: 67bc700498 user: [email protected] tags: main, trunk

A horizon given through CloneOpts.Horizon or --horizon now always goes out as pragma horizon-name, a full hash at depth 0 included. The origin tries a full hash as a check-in and then as a symbolic name (§6.2.7, C.3), so a branch or tag spelled as 40 hex digits is found; a peer that resolves no name is covered by the local fallback. A horizon held on the store cone still goes out as pragma horizon on every pull, sync and deepen.

nameAuthority now says why crediting the login card to the whole request is harmless, and TestResolveNameBounded fails within 2s with a message naming the unbounded walk. check-in: 017c7f9393 user: [email protected] tags: main, trunk

fix(sync): ask for phantoms in hash order check-in: d40046f706 user: [email protected] tags: main, trunk

buildGimmeCards ranged over the phantoms map, so each round's gimme cards came out in Go's randomised map order. When a round has more phantoms than mxPhantomReq allows, that also picked a random subset to ask for. A seeded simulation then could not replay the same wire, and the docs generator's engine.json (which records a pull's cards) changed between runs, failing the generated-docs check about one run in three.

Sort the phantom names before building the cards. Phantom sets are bounded by what the peer announced, so the sort is cheap next to the round trip. check-in: 82dae665dd user: [email protected] tags: main, trunk

An origin resolved a horizon-name before the requester's login was read, and served a requester that may neither read nor clone at the horizon the name resolved to, so whether a name exists showed in its reply. The login card is now checked first, and only a requester that may read or clone has the name resolved; anyone else is answered as if the request named no horizon, the same reply for any name. A pull naming an unknown horizon is refused with the one error card, not served densely.

The client fails the clone on a second horizon-resolved in a reply, and on one answering a request that sent no horizon-name. A private origin's challenge does not settle the name: the next request resends it with the login, still the clone's first request under §6.1.

API: CloneOpts.Horizon replaces CloneOpts.HorizonName and the stand-in check-in a named horizon's StoreCone had to carry. It reuses the Horizon type, whose Checkin may there be a name, beside the existing HorizonDepth, and keeps the rule and the structural level where they were, so no public concept is added. StoreCone gives the path members alone: a horizon on both, or a depth without a Horizon, is an error. A full hash at depth 0 goes on the store cone as itself. Internally a NamedHorizon carries the name, depth, rule and level, not a cone holding a placeholder hash, and only the resolved hash is ever stored. The CLI passes --horizon through the same field. check-in: 54d75fbc78 user: [email protected] tags: main, trunk

ResolveName stepped back one query per check-in, up to 2^32-1 times. A recursive query now walks the primary parents at most depth steps and never more than the check-ins the repository holds, so even a corrupt cycle of primary parents ends. check-in: 6f12912a01 user: [email protected] tags: main, trunk

Two check-ins tied on mtime resolve to the lexicographically lowest full hash, never the lower local rid, so two origins, and a clone resolving locally, resolve a name alike (§6.2.7 clarification C.2). HZ-6.2-17 and the horizons page say so. check-in: c941f6177b user: [email protected] tags: main, trunk

CloneOpts.HorizonName and HorizonDepth, and repo clone --horizon NAME --depth N, ask the origin for the horizon by name; CloneResult reports the resolved check-in, the steps taken and whether the clone resolved it locally. A full hash with no depth is still sent as the hash itself. Docs and the generated CLI reference and conformance data follow. check-in: 94bff91c0d user: [email protected] tags: main, trunk

An origin resolves pragma horizon-name NAME RULE DEPTH over its dense repository, echoes pragma horizon-resolved HASH STEPS ahead of horizon-served and every file card, and serves the request as if it named the hash, minting any structural seed for that check-in. A name that resolves to nothing, a malformed depth, or both pragma forms in one request is answered with one error card, after any login or capability refusal.

A clone sends horizon-name only until the name settles, takes the echoed hash as its store cone's horizon, and fails, leaving nothing on disk, on a malformed, unasked-for, late or over-deep horizon-resolved or a check-in that never arrives. Against a peer that answers none it finishes the dense transfer and resolves the name locally at the content horizon.

Conformance rows HZ-6.1-2, HZ-6.2-17, HZ-6.2-18, HZ-6.3-3 and HZ-6.3-4, and HZ-3.4-2 as amended, are written from the erratum ahead of the draft's re-pin, with the nine named-horizon cases at both ends plus the malformed-request and client-check cases and a stock-fossil fallback. check-in: 1f4a69f2df user: [email protected] tags: main, trunk

ResolveName turns tip, a symbolic name or a full check-in hash into C as draft-fossil-horizon-00 §6.2.7 (zit 0.7.1 erratum) does: tip is the most recent check-in, a full hash names its check-in, and any other name is the most recent check-in carrying an active sym-NAME tag, ties going to the lower rid. It then steps back along primary parents, stopping at a root, and reports the steps taken. check-in: ee8a559b4e user: [email protected] tags: main, trunk

build(docs): one deploy build, through scripts/docs-build.sh check-in: 5526734cfd user: [email protected] tags: main, trunk

The deploy ran two builds: the Cloudflare trigger's build command and wrangler.jsonc's own. The triggers now have none, so wrangler.jsonc is the one place the deploy build is defined, and it calls the same script developers run. check-in: 235cf4fd8d user: [email protected] tags: main, trunk

docs: rework the site and landing page, generate its facts from the code check-in: 4caaddcd19 user: [email protected] tags: main, trunk

The Build (tinygo) job builds examples/tinygo on every change, so its size (3.52 MB raw, 1.26 MB gzipped) is the one the site quotes; the README keeps the macOS measurements beside it. check-in: 313f64a89e user: [email protected] tags: main, trunk

feat(wasm): build the whole public API with TinyGo check-in: b3680c4757 user: [email protected] tags: main, trunk

go-libfossil compiled for js/wasm only with the standard toolchain, whose output is about 16 MB, too large to ship with a page or an edge function. Three standard-library features TinyGo lacks kept it from building there: http.ErrUseLastResponse and net.DNSError in the sync client, and os.SameFile in checkout-marker recovery.

Each now sits behind a tinygo build tag. Under TinyGo a sync redirect ends the request with an error instead of being followed, DNS failures are not classified as transient, and two names of one checkout marker are reported as ambiguous rather than paired; the standard build is unchanged.

examples/tinygo exercises the whole public API, sync and signing included. Built with TinyGo 0.43 and wasm-opt -Oz it is 3.83 MB raw and 1.40 MB gzipped, against 16.1 MB and 4.06 MB from the standard toolchain. A new CI job builds it with TinyGo on every change. check-in: 0650171d55 user: [email protected] tags: main, trunk

go-libfossil now builds with TinyGo, sync and signing included (examples/tinygo). The landing page reads the measured sizes from data/wasm.json instead of an outside project's engine-only build, and the install page says how to build it and what differs under TinyGo. check-in: 7377cb7c97 user: [email protected] tags: main, trunk

Runs sync-version.sh, gen-llms-txt.sh and Hugo, given the hugo binary as its first argument. It needs no Go toolchain: the inputs that do are committed and checked for freshness in CI. make docs-build uses it. check-in: 1ef14e7167 user: [email protected] tags: main, trunk

The home page renders its hero figure, conformance table, version and footer from the generated data and site params. Docs pages keep Hextra with Fraunces, Inter and JetBrains Mono and rust as the accent in both themes. Add the get-started pages (install, CLI quickstart, embed in Go, first sync), the conformance reference rendered from data, and point the nav, README and old links at the new layout. check-in: 41dfd5c149 user: [email protected] tags: main, trunk

tools/docsgen/engine records one check-in as an artifact, as its derived rows and as the cards of a pull, from a real engine run with fixed inputs. tools/docsgen/conformance turns TestCoverage's report into per-draft and per-requirement data. scripts/gen-cli-docs.sh writes one page per command group from the libfossil binary's --help. gen-llms-txt.sh and check-llms-txt.sh are ported from dagnats, and sync-version.sh now takes the highest release tag and writes the go.mod version too.

scripts/check-docs-generated.sh regenerates everything into a scratch dir and fails on any difference; make docs-check and the test-docs CI job run it. check-in: d7fd910ffc user: [email protected] tags: main, trunk

Move the flat docs pages into sections, keep each old URL as an alias, split sparse cones from horizons, and add concept pages for artifacts, repository state, sync, sub-manifests and signing plus a settings reference. Audit the moved pages against the code and fix drift. check-in: 93ad669695 user: [email protected] tags: main, trunk

feat(signing): sign check-ins and horizon seeds with gpg or ssh check-in: 4a439c6de6 user: [email protected] tags: main, trunk

go-libfossil signed nothing and verified nothing, so a structural-horizon clone could not tell a seed its origin minted from one substituted on disk (draft-fossil-horizon-00 §5.2.4, HZ-5.2-5 and HZ-5.2-6), and a repository with fossil's clearsign setting on stored its check-ins unsigned.

Signing follows fossil's contract, the same one zit follows. With clearsign on (and the legacy omitsign off), every check-in a commit or branch creation stores is clear-signed by pgp-command: unset, gpg --clearsign -o, run as COMMAND OUTFILE INFILE; a command named ssh* runs as COMMAND INFILE and its detached signature is wrapped in fossil's SSH envelope. A signer that fails, or returns text that does not strip back to the manifest, abandons the commit. NoSign and --nosign store one check-in unsigned.

An origin with clearsign on signs each seed it mints. It signs only after the request is authorized, outside the request's transaction, keeps one signature per seed and command so the seed's name is stable, and re-signs a kept signature its own horizon-seed-key no longer verifies. Keeping is best effort, so a read-only mirror still serves.

A clone given its origin's key (CloneOpts.SeedKey, repo clone --seed-key, kept as horizon-seed-key) verifies every seed on clone, pull and rebuild, and falls back to the content horizon from one unsigned or signed by another key. An SSH key is verified in process: namespace fossilscm, the configured key, no SHA-1 ssh-rsa, and exactly fossil's envelope. An OpenPGP fingerprint is verified by gpg, which must report one good signature by that key over text byte-equal to what the parser reads, with nothing after the signature. A verifier that cannot run keeps the held seed and fails the sync instead of discarding it.

fossil 2.28 and zit read the signed check-ins; a check-in zit signs with the same SSH key verifies here; zit clones a signing origin and holds its seed. check-in: 612f2e7acc user: [email protected] tags: main, trunk

chore(release): v0.15.0 check-in: a992dcb34f user: [email protected] tags: main, trunk, v0.15.0

feat: horizon cones (draft-fossil-horizon-00) check-in: ac0bf85d72 user: [email protected] tags: main, trunk

The previous change let a sparse store link a check-in before its files. A pull that deepens a structural horizon then linked the new base first and left its files unrequested phantoms, and the merge over it found no content (TestRepoMergeDeepensAcrossHorizon).

A pull waits on a check-in's files again, except those the store cone's path members exclude, which never arrive. Only a clone into a sparse store waits on none: the origin's walk sends every file the store keeps in the same session, and nothing reads the store before the clone ends. check-in: 35566881ec user: [email protected] tags: feat/horizon-cones, trunk

A sparse store published no event or plink rows for a check-in while a P or Q source of it was absent, unless a structural horizon held it below. A source can be absent for good: fossil-scm.org's repository lacks the parents of five check-ins, which a dense clone links against phantoms, so a cone or content clone of it failed repo verify on them.

SparseCheckinReady no longer requires an absent source's file set: the check-in's spine is published against the phantom, as a dense store publishes it, and its orphan row keeps mlink derivation waiting until the source arrives. check-in: 69f3048e67 user: [email protected] tags: feat/horizon-cones, trunk

A cone-aware clone charged each row it walked past at its expanded size, while a dense round charges the bytes it sends, a row's stored delta. A sparse round of fossil-scm.org's repository covered a hundredth of a dense round's rows and the clone ran out of rounds. SC-6.2-1 bounds the walk as a dense clone bounds its batches, so a walked-past row is now charged its stored size, read from the four-byte prefix of blob.content without inflating it. check-in: 835557d4b1 user: [email protected] tags: feat/horizon-cones, trunk

A cone-aware clone classifies each row it walks, and each served delta's base, by parsing the row's bytes. The handler has no content cache in production, so every row's whole delta chain was expanded and hashed again; on fossil-scm.org's repository that was the cone clone's first round, unfinished after 20 minutes.

The walk now reads, once per request, the rows the crosslink tables prove to be file content (named by mlink, claimed by no artifact table; a store that has seen a sub-manifest answers nothing) and classifies only the rest, through its own unchecked cache: classifying serves nothing, and the bytes a row is sent with are still checked apart.

A file version's base is the newer version, which a walk in rid order has not reached yet, so every version went out whole: a 9 MB text per version of sqlite3.c. A content base the walk has not reached is now forwardable when the store's classification of the cone keeps it (SC-6.2-2 bars only a base outside the keep set). check-in: 9a2938b626 user: [email protected] tags: feat/horizon-cones, trunk

A clone's receive linker tested a deferred check-in's every F card again each time one blob it waited on arrived, and each time a manifest it was a delta of arrived, and its waiter index (8192 entries) filled after a few check-ins of a large tree. A sparse store's out-of-cone files never arrive, so a cone or content clone of fossil-scm.org's repository spent its first round, past 20 minutes, in availability queries.

A sparse store now waits on no file: it holds what its cone keeps, and SparseCheckinReady judges its completeness. A waiter strikes each arrival from its wait set and is examined again only once the set is empty, and a check-in already waiting is not re-examined when a delta source arrives under it. check-in: 09b62091ec user: [email protected] tags: feat/horizon-cones, trunk

A check-in crosslinked before its cherrypick ('+') source arrived kept no cherrypick row, since insertCherrypicks skipped a source it could not find, and so never received the auxiliary mlink rows for that source: the late-arrival pass walked plink alone, which holds no cherrypick edges. Canonical skips a missing source the same way and relies on the rebuild its clone ends with; a clone here links as artifacts arrive and has no such rebuild. A clone of fossil-scm.org's repository was left with 139 cherrypick rows and 262 mlink rows fewer than its origin.

insertCherrypicks now reserves a missing source as a phantom, as plink does for a missing parent, and the late-arrival pass (linkPlinkChildren, now linkLateChildren) walks the source's cherrypick children with plink's. check-in: 28f343c11b user: [email protected] tags: feat/horizon-cones, trunk

forwardableSource reads a row, its shun and private state and the tree gate each time it is asked, and a clone round asks about the same rows from every chain walk and every sentEarlier search: a twelfth of a dense clone's CPU. A request reads one snapshot, so the verdicts are kept for its duration.

TestCorpusClone logs an origin error, which its client sees only as an HTTP status. check-in: b9ec3c086e user: [email protected] tags: feat/horizon-cones, trunk

ReceiveLinker.LinkStored made a fresh availability cache for every blob it linked, so each deferred check-in's retry walked every F card's delta chain again; after the receive fixes that walk was a sixth of a dense clone's CPU. Only the unavailable verdicts can go stale during a receive, which only adds content, and a clone whose round fails stops receiving, so LinkStored now drops just those (AvailabilityCache.ForgetUnavailable) and keeps the rest. check-in: a30155185d user: [email protected] tags: feat/horizon-cones, trunk

A clone round sends each row with its delta chain, source first, but only remembered what it sent within the round. fossil deltifies each version of a file against the next, so the rows of a file's versions come in rid order, each a delta on a later one: the oldest version's round sent the whole history, and each later version's row, reached in a later round, sent the rest of it again. Cloning the Fossil repository received 333,063 delta cards by round 64 for 22,353 new artifacts, each checked again by the client.

The origin now infers what earlier rounds sent: a row below the cursor that its round sent as itself reaches, through sources this clone forwards, every node of its chain. sentEarlier walks down the delta table from a row or a chain source, at most 256 rows, and a row it finds sent is skipped like one sent earlier in the round, and a chain stops at a source it finds sent. Not finding one sends again, which is only redundant. A row below the cursor filled or published after its round passed it is taken as sent; rows below the cursor are not sent again in any case. check-in: 07b30dccde user: [email protected] tags: feat/horizon-cones, trunk

The real-repository gate found a dense clone of the Fossil repository CPU-bound past an hour, in delta.Checksum and chain expansion. checkDelta rebuilt each received delta's basis from the root of its chain with content.ExpandUnchecked, after walking the chain for availability; a file's history arrives as one chain of hundreds or thousands of deltas, so the cost was quadratic in its depth, as Repo.Verify's was before 1a5abca.

A receive pass now rebuilds the basis through an unchecked content cache (content.NewUncheckedCache): from the deepest node of the chain it holds, keeping each node it rebuilds, and it remembers every artifact a card stored and verified once the card's savepoint commits. A clone keeps one such cache across its rounds and clears it when a round rolls back. A basis the cache holds, or one dangling in the pass, needs no availability walk, and the clone's check for a basis to request uses the same knowledge. The cache holds what each row rebuilds to, unchecked as checkDelta's basis always was, so every verdict is unchanged; a verified delta's target also goes to the receive linker, which no longer expands it again.

TestCorpusClone (LIBFOSSIL_CORPUS) clones a copy of a repository from an in-process server for profiling. check-in: b43fd24a1f user: [email protected] tags: feat/horizon-cones, trunk

Every request ran in a write transaction (BEGIN IMMEDIATE) for as long as it was served, a horizon request read the classification memo in another, and each request created the table-sync schema on a repository without it. A clone arriving while another request was served waited for the write lock past the 5s busy timeout and got HTTP 500 "database is locked (SQLITE_BUSY)"; its client gave up after four attempts. The real-repository gate hit it with a horizon clone beside a dense one.

A request none of whose cards can write now runs in a read transaction (db.WithBoundReadTx), which takes a snapshot without the write lock; the memo read does too; loading synced tables only reads, and a schema card creates the schema it registers into. Requests that write keep their write transaction. v0.14.0 has the same write transaction per request: a clone from its server fails the same way while another connection holds the lock. check-in: f8bb98e196 user: [email protected] tags: feat/horizon-cones, trunk

An origin answering a horizon request never sends a cluster, even one a gimme names; an origin may name a history URL in horizon-served (horizon-history-url), which the client reports; and a merge with no peer to ask refuses whenever either side's ancestry crosses the horizon. Conflicts in the changelog and the CLI horizon tests kept both sides. check-in: c4b191ed05 user: [email protected] tags: feat/horizon-cones, trunk

draft-fossil-horizon-00 §5.5 has a structural clone ask the origin for the base whenever either side's ancestry crosses the horizon, even when its own walk finds a common ancestor: over part of the graph that ancestor need not be the nearest. Without a peer Checkout.Merge and Repo.Merge refused only when the walk itself crossed, and merged over a held base whose sides' ancestry crossed below it.

Both now refuse such a merge with an error matching ErrBelowHorizon that names the horizon and points to Checkout.MergeContext with a Peer; MergeContext refuses the same way when the origin does not answer. A merge whose ancestry stays above the horizon, or one given its base, still merges locally. The CLI's merge already falls back to asking the remote on ErrBelowHorizon, so it now asks in these cases too. check-in: 263de5a736 user: [email protected] tags: feat/horizon-cones, trunk

The horizon draft pinned at zit horizon-rfc 5d618aaeb1: an origin with no tag in force mints no seed and serves the content level, and choosing C is shown with each rule's cost (repo cone cost, Repo.HorizonCost). check-in: 77e74bd011 user: [email protected] tags: feat/horizon-cones, trunk

Draft §3.4 asks a tool that suggests a horizon to show, before the choice, what each rule would hold in check-ins and bytes. repo cone cost <check-in> prints that for descendants and leaves from the local repository, and Repo.HorizonCost returns the figures. The --horizon help of repo clone and repo cone store points to it, noting that a clone has no repository yet. check-in: 834b28d9fe user: [email protected] tags: feat/horizon-cones, trunk

The draft now says an origin mints no seed, and serves the content level, when no tag is in force at any check-in the seed would cover, since a control artifact needs a T card. HZ-5.2-2 loses its scope note and the new HZ-5.2-7 row is proven by a case that mints none, advertises horizon-served content and leaves the clone holding no seed. HZ-3.4-1 is restated as guidance for the tool; it is covered with the cost report in the next commit. check-in: 4e9dea8cc1 user: [email protected] tags: feat/horizon-cones, trunk

Annotate in a structural store marks the lines that reach the horizon and names the deepen that would show their origin (§5.5); the origin memo keeps the trees a horizon's walk reads in a bounded cache, so a second horizon over the same store reads only trees the first did not. check-in: 479cbaad12 user: [email protected] tags: feat/horizon-cones, trunk

draft-fossil-horizon-00 §6.2 asks an origin to read each structural artifact's references once per store and classify every cone over that graph. The memo did so for kinds and P, B, S and A cards, but the F cards of a horizon's trees were walked from the store by the first classification of each horizon, so a second horizon under another C or rule re-read every tree it shared with the first.

The memo now keeps, per store lineage, each manifest, baseline and node a walk read: its own F-card targets and its B and S references, so a delta manifest shares its baseline's entry. A view stages what it reads and the memo takes it on Publish, so a request that rolls back leaves nothing; a kept piece serves a view only when its rid is held at that view's prefix, and its size is charged as a read would charge it, so budgets fail where a fresh walk fails. Names are interned and the cache is capped at 64MiB with LRU eviction: fossil-scm's history names about 21M F entries, over 300MiB uncapped.

On a copy of a 21k-check-in fossil repository, horizon B (2000 check-ins back) classified after horizon A (1000 back) took 2.5-3.0s and now 1.6s, reading 995 trees instead of 1995; A is unchanged at about 1.3-1.6s. check-in: 13f8021952 user: [email protected] tags: feat/horizon-cones, trunk