Timeline

A hosted SQLite that caps value size (a Durable Object's 2 MB) cannot hold a large file's stored blob in one blob.content value. With the new db.DriverConfig.MaxValueBytes set, a stored value over the limit is written in 1,000,000-byte parts to blobpart(rid, seq, content), created on first use, leaving blob.content NULL and blob.size the expanded length, so phantoms (size < 0) stay unambiguous. Chunking sits below compression: hashes, deltas and the wire format are unchanged.

internal/blob owns the seam (ReadStored, WriteStored, InsertStored, StoredHead, DropStored, SweepParts and two scan expressions), and every direct reader or writer of blob.content now goes through it: delta rewrite and undelta, receive, clone send and walk, compaction, sparse demotion, seed drop, shun purge, horizon cost and cone preparation.

Zero, the default, keeps every value inline and creates no table, so local stores stay byte-identical. A store with blobpart rows is not readable by stock fossil; enable it only for hosted backends.

Closes #459 check-in: 1fd12b3ed3 user: [email protected] tags: feat/chunked-blobs, main, trunk

conformance: HZ-4.1-1 scoped to zit and go-libfossil origins check-in: 1455a3803e user: [email protected] tags: main, trunk

The origin classification memo now reads the F cards of every tree in the store once, with its references, and extends them by appended check-ins (draft-fossil-horizon-00 §6.2, HZ-6.2-10). Blob names are interned to uint32 ids, each tree's file set is a sorted delta-varint id list, and a full manifest is kept as its symmetric difference from the previous full manifest read, at most 15 links from a piece kept whole, decoded through a 16-entry LRU per view. A store whose index would pass memoTreeBytes keeps none and falls back to the bounded cache of walked trees.

On a clone of fossil-scm.org the trees take 11 MiB (the walked-tree cache held 50 MiB for two horizons), the memo's retained heap halves and peak in-use heap drops from 345-524 MiB to about 170 MiB. Tracked in #463. check-in: 1cb9e11a65 user: [email protected] tags: main, perf/memo-tree-encoding, trunk

Drop the stock-fossil caveat from the row, so it reports COVERED, and probe zit's reply to req-clusters under a horizon pragma (after a dense control). leaf check-in: e9eb2cabac user: [email protected] tags: conf/hz-4-1-1-zit-scope, main, trunk

perf: carry a cone clone's forwarded delta chains across rounds check-in: b85673a6cb user: [email protected] tags: main, trunk

fix(content): classify a fossil-written store's file versions from its tables check-in: d3574f0c74 user: [email protected] tags: main, trunk

test: expect fossil's newest-first ticket history check-in: 3ef7f9fb1f user: [email protected] tags: main, trunk

The mlink-named, unclaimed versions are taken as file content in a store with the non-artifact record too, including this library's own between a Checkin that commits an artifact-shaped file and the sweep that links it. The comment now says so, and that the window closes as the sweep's batches add the claims the memo follows. check-in: dd3f875e99 user: [email protected] tags: main, perf/cone-clone-rounds, trunk

The re-deltified version now sits inside a carried run, and the unreached blob it is stored against is itself a delta of the run's tip, so the chain from the run's head keeps the run's length and differs only in its rids. Disabling the checksum in cloneRun now ends the clone with a phantom. check-in: caea59079f user: [email protected] tags: main, perf/cone-clone-resend, trunk

fossil ticket history orders changes by time, newest first (tkt.c, ORDER BY 1 DESC), and leaves a same-second tie unordered. The test expected the original ticket before its amendment, which held only when both landed in the same second and the tie broke that way; CI failed it whenever the clock ticked between them.

Stamp the amendment a minute later and expect it first. leaf check-in: 88082202ad user: [email protected] tags: test/ticket-history-order, trunk

seedcheck: drop a seed completed as a delta in a later round check-in: 93cf3b65f6 user: [email protected] tags: main, trunk

A delta on a refused seed now counts as a new phantom only when it makes one, so a peer resending the same delta no longer holds a pull open to the round cap. The phantom it leaves is private when the card was, so its name is never asked for in a public sync.

seedcheck's drop reuses db's savepoint instead of its own copy: db.Tx.WithSavepoint exposes what WithTx already does on a bound DB. check-in: f8446cdfcc user: [email protected] tags: fix/seed-delta-drop, main, trunk

A push into a store fossil wrote starts the non-artifact record with what arrived, and loadFiles then took only the record's versions as file content: every older version went back to be read raw, past the budget on a real repository. File versions are now the record's together with those mlink names and no claim table names (fileVersionsSQL), in a full load and in a memo's extension alike, and the memo's prefix counts the claims in every store, since file versions follow them whether or not a record exists. The horizon memo conformance case that pushes a manifest ahead of its blob into such a store now extends by the arrivals again. leaf check-in: 9cc2297ef8 user: [email protected] tags: main, perf/cone-clone-rounds, trunk

A request carries a cookie of any length, and splitting one on ':' or '.' before any bound allocated a string header per separator: 8 MB of dots cost hundreds of megabytes. cloneCapFields now refuses a cookie past maxCloneCookieBytes (64 KiB, well past the longest the server writes) before it splits, and splits with SplitN, as parseCloneRuns does.

A forged cookie could also name one deep chain maxCloneRuns times. The server now rebuilds each head once and at most maxCloneRunRIDs rids in all, newest runs first, and the cookie keeps the newest maxCloneRuns runs rather than the oldest, since an old run may never be reached.

A SHA3 artifact refused to an older peer was recorded as sent, and so carried in a run across rounds; it is now left out as a withheld tree is.

TestConeCloneSurvivesRedeltify re-deltifies a forwarded version against an unreached in-cone blob mid-clone and checks the clone ends whole; it fails with a phantom if the run's checksum is not checked. leaf check-in: 624df80df2 user: [email protected] tags: main, perf/cone-clone-resend, trunk

Without the non-artifact record, file versions are those mlink names and no claim table names, and a sweep linking an artifact late adds claims without adding a blob. The memo's store prefix now also counts the rids the claim tables name at or below its top (claimRows; distinct rids, so a check-in's propagated tags do not move it), which settled checks before extending, and an extension takes as file content the old versions the arrivals' mlink rows name that no claim table names, as it does with the record.

loadFiles' comment now says what the fallback decides rather than claiming a proof: an unlinked blob that parses as an artifact is the file mlink says it is, elided outside the cone, and a corrupt out-of-cone version is elided unread. Tests pin both. check-in: 40b018695a user: [email protected] tags: main, perf/cone-clone-rounds, trunk

With the store classified, the cone clone of fossil-scm.org's src takes 8 rounds and receives 57,041 blobs to hold 56,262, against 12 rounds and 83,521 without the carried chains. The chains also cover versions a merge check-in brings in from a branch, which made up most of the remaining repeats. check-in: 102dab787b user: [email protected] tags: main, perf/cone-clone-resend, trunk

Review follow-ups to the seed drop.

A delta that arrives on a seed already dropped could never be read: it was stored on a content-less row that is never filled or asked for. It is now not stored as a delta but left a phantom and asked for again in full, on a pull, a clone's phantom pull and a push to an origin. A seed that arrives in full after deltas built on it is stored just long enough to check those deltas, judge them and rewrite them as full text, and then dropped.

seedcheck.Drop runs in a savepoint (or a transaction of its own on a pool), so a failure partway, which the after-dephantomize cascade logs and commits past, leaves the seed and its dependents as they were.

An origin answering a push no longer asks for a refused seed's phantom, and a clone asking for what it left missing no longer counts one. fossil 2.28's rebuild re-adds every size<0 row to the phantom table and deletes the store-local seedrefused and seedpending tables; the CHANGELOG notes this and the one-time drop of refused seeds that older stores kept.

The client now adds to a reply's new-phantom count rather than replacing it, so a phantom made while receiving a file continues the exchange. leaf check-in: 41c092205e user: [email protected] tags: fix/seed-delta-drop, main, trunk

A store this library crosslinked records each blob it found to be no artifact, and classification skips the file versions among them. A store fossil wrote has no such record, so classification read every file version raw and exceeded its byte budget: on a copy of fossil-scm.org's repository it failed on every request. A cone clone's walk then kept no base it had not reached yet, sent every in-cone version whole, and fit two or three 2-4 MB sqlite3.c texts into a 16 MB round: about 390 rounds and 30 minutes, where a dense clone takes 8 rounds.

Without the record, a version that mlink names and no table an artifact's crosslink writes claims is file content too, the proof the cone walk's knownFile already relies on; a table the store lacks claims nothing. That clone now takes 12 rounds and 9 minutes, verifying the same.

Related to #465. check-in: 4536dae44c user: [email protected] tags: main, perf/cone-clone-rounds, trunk

The server's cone walk sends an in-cone file version as a delta with the chain of newer versions ahead of it as its bases. Each round restarts the walk from the clone's cursor, so every later round that reached one of those versions at its own check-in sent it again.

The walk's cookie now names each chain a send forwarded: its head, length and an FNV-64 checksum of its rids. The next round rebuilds the chain from the store's delta table, counts its rids as sent, and drops it when the checksum no longer matches, so a store re-deltified mid-clone can never pass off a base the client was not sent. A run loses the leading rids the walk reaches, and the cookie carries at most 1,024 runs; one dropped is only sent again.

On a copy of fossil-scm.org's repository a cone clone of src now receives 72,835 blobs to hold 56,262 (main: 100,212); the clones verify identically. check-in: 9d266a0f51 user: [email protected] tags: main, perf/cone-clone-resend, trunk

A horizon seed stored as a delta whose base arrived only in a later round (pull, clone) or a later request (origin push) was refused by its hash but kept in the store, and a pull or clone counted the delta as received on arrival, before it could be judged (draft-fossil-horizon-00 §5.2.3, HZ-5.2-14).

On a seed verdict seedcheck now rewrites each delta built on the seed as full text, then empties the seed's row: size -1, no content, no delta link, and kept out of the phantom table so no request asks for it. The rid stays held for anything a receive pass tracks by it, and the refused hash is still recorded in store-local state.

A pull or clone counts a delta it could not read on arrival only at the end of the exchange, once readable and not dropped. A round still treats it as progress. The receive pass treats a row without content as no basis even when its cache holds the old bytes, and the push handler's same-request seed check consults the refused hash, since the dropped row no longer reads.

HZ-5.2-14 is now covered. check-in: 917bcc9830 user: [email protected] tags: fix/seed-delta-drop, main, trunk

test: move black-box tests to apitest/, consolidate root tests by source file check-in: b4ac68be08 user: [email protected] tags: main, trunk

ci: cache the pinned Fossil and zit release tarballs check-in: 38eb1f810c user: [email protected] tags: main, trunk

fossil-scm.org answered 503 for the Fossil release tarball, and both test jobs failed before any test ran. The checksummed tarballs are now cached by version and checksum, so fossil-scm.org and zit's server are needed only on a cache miss; a cached copy is still verified before it is installed. check-in: b65ca240f1 user: [email protected] tags: main, trunk

Merge the root package's white-box test files so each mirrors the source file it exercises (checkout, fossil, horizon, repo_checkout, repo_history, repo_merge, repo_sync, repo_ticket, repo_extras, required_content, transport), and rename single files to match their source (repo_content_cache_test.go, repo_admin_test.go, required_content_checkout_test.go). Test bodies are unchanged; only imports are consolidated. check-in: af6431af27 user: [email protected] tags: main, trunk

Every package libfossil_test file in the root used only exported identifiers and shared no helpers with the white-box tests, so all 21 move to a test-only apitest package. Docs that name their location follow. check-in: 6956a92661 user: [email protected] tags: main, trunk

feat(sshverify): make in-process SSH signature verification opt-in check-in: 5e7ce934cd user: [email protected] tags: main, trunk

# Conflicts: # CHANGELOG.md check-in: 52d7948375 user: [email protected] tags: main, trunk

simio is the third package an embedder links; the opt-in guard now checks it alongside the library and db. check-in: d79a35c16a user: [email protected] tags: main, trunk

feat: ticket writes, ticket reads and timeline-since check-in: e448e2e48d user: [email protected] tags: main, trunk

internal/clearsign keeps the SSH envelope, SSHSIG parsing, namespace, hash and SHA-1 checks, none of which need cryptography, and hands the key parse and signature check to an SSHVerifier that the new public package sshverify registers from init. The library therefore no longer links golang.org/x/crypto/ssh and the ECDSA, P-256, Ed25519 and RSA code it pulls in; html-artifact-publisher's TinyGo engine drops from 1,347,569 to 1,093,647 bytes gzipped (3,777,936 to 3,208,411 raw).

Without the verifier nothing is accepted unchecked: a well-formed SSH signature is ErrSSHVerifierUnavailable, which wraps ErrCannotVerify, so a seed under an SSH horizon-seed-key is refused, the held one is kept, and the sync or rebuild fails naming the package to import. Malformed signatures are still bad. The cli package (and so cmd/libfossil) and examples/tinygo import sshverify; a go list guard keeps x/crypto/ssh out of the root and db closures. check-in: a87790f468 user: [email protected] tags: main, trunk

# Conflicts: # CHANGELOG.md check-in: 1d8b2e6658 user: [email protected] tags: main, trunk

fix: let Create proceed when the filesystem cannot stat the parent check-in: b7e00e1673 user: [email protected] tags: main, trunk

Create's parent-directory check, added in #457, refused a path whenever os.Stat of its parent failed. Under TinyGo wasm a repository lives in a SQLite VFS and os.Stat is unsupported, so every create failed with "parent directory does not exist: /"; standard Go's js/wasm build hung on the same check. It now refuses only a parent the filesystem confirms is missing or is not a directory, and otherwise lets SQLite's own open report a real failure (tracked in #509). check-in: 2449bcd313 user: [email protected] tags: main, trunk

A sync handler asks what a push brought in. Filtering on timestamps missed an artifact made on a machine whose clock runs behind the given check-in's. TimelineSince now keeps entries whose artifacts were stored after the given one: a later receipt (blob.rcvid, which a phantom fill updates), then a higher rid. Any held artifact hash is accepted; an unknown one is an error.

Tracked in #504. check-in: 29b53e75d2 user: [email protected] tags: main, trunk

Repo.TicketAdd and Repo.TicketSet write the ticket-change artifact stock `fossil ticket add` / `fossil ticket set` write (D, J incl. +append, K, U, Z), byte for byte, and crosslink it in the same transaction so the ticket, ticketchng and event rows read back at once. TicketOpts.Comment is stored as the stock ticket forms store a remark: icomment with login, username and mimetype.

Repo.TicketList, Repo.TicketChanges and Repo.TimelineSince expose the reads the html-artifact-publisher engine needs, as JSON-ready values. manifest.Timeline gains Kinds and Since to back TimelineSince.

Tracked in #503 and #504. check-in: 68f2c3d930 user: [email protected] tags: main, trunk

chore(release): v0.16.2 check-in: 89f09bf179 user: [email protected] tags: main, trunk, v0.16.2

test(conformance): close four partial horizon rows check-in: 94717e5803 user: [email protected] tags: main, trunk

HZ-5.2-2: the seed label appears on no artifact an origin or its structural clone stores, across re-seeding and writes on both sides; the minted floor survives a restart of the origin. HZ-5.2-9: a seed's missing base filled by a pull, a local check-in and a local tag leaves the seed refused, unlinked and unserved, and the transfer that filled it linked. HZ-6.1-2: path-cone and cursor pragmas beside horizon-name are answered as beside pragma horizon, and act; a named clone with path members sends them and records both. HZ-6.2-17: an event-mtime tie resolves to the lowest full hash for tip and a branch; a full-hash-shaped name no check-in has resolves as a symbolic name, and one a check-in has names that check-in first. check-in: 7a375b6d73 user: [email protected] tags: fix/conformance-subtest-t, main, trunk

test(conformance): hand each case its subtest's t; cancel stale Race runs check-in: 49bf242db8 user: [email protected] tags: main, trunk

check runs each case as a subtest, but the suites passed the case the parent's t, captured in a closure. A fixture that skips or fails, as the fossil-built R-card fixture skips when fossil(1) is absent, then called SkipNow or FailNow on the parent from inside the subtest, which Go fails as "subtest may have called FailNow on a parent test". The Race workflow installs no fossil and failed this way in TestSparseCone. check now hands conforms the subtest's t, and all 150 call sites take it, so a case's skip or failure ends that case alone. TestCheckSkipEndsOnlyTheCase guards the harness.

A Race run takes most of an hour and merges land faster, so a new run on a ref now cancels the older one. check-in: 43df2fd2c2 user: [email protected] tags: fix/conformance-subtest-t, main, trunk

docs: regenerate the SDK reference and check it in CI check-in: 51d3b9cf5f user: [email protected] tags: main, trunk

The gomarkdoc SDK pages had drifted about a thousand lines from the code: make docs-gen-sdk was never part of docs-gen, so nothing regenerated them and the docs check did not compare them. ReadBlob, SetContentCache, ListFilesIn and OriginNamer were missing from the libfossil page.

scripts/gen-sdk-docs.sh now renders them, running gomarkdoc v1.1.0 through go run so CI needs no install, then adds the frontmatter. It honours OUT_DIR, gen-docs.sh calls it, and check-docs-generated.sh compares the result, so a stale SDK page fails CI like any other generated input. make docs-gen-sdk calls the same script. llms.txt and llms-full.txt are regenerated from the refreshed pages. check-in: c12b02189f user: [email protected] tags: main, trunk

draft-fossil-submanifests-00 §7.4: enumeration results SHOULD be cached keyed by artifact name, so a rebuild parses each distinct sub-manifest once. The tree walk's loader now serves a resolved node from, and keeps it in, a new byte-bounded LRU memo on content.Cache (DerivedSized), keyed by the node's hash and charged its bytes, within a quarter of the cache's budget. The existing 32-entry Derived FIFO is sized for a few parsed check-ins; a tree's working set is every distinct node, which would thrash it and evict those check-ins.

Only a node that resolved is kept, so a phantom is looked up afresh and fails the walk as before. Depth, cycle and node-count bounds stay per walk. The memo is dropped with Derived on Invalidate and Clear, and the Repo's cache is replaced at every write.

Listing a tree that differs from one already read in one leaf now parses its manifest and that leaf; listing every check-in of zit's recorded tree history parses 48 sub-manifests instead of 109. SM-7.4-1 is covered.

Tracked in #493. check-in: 94f61db228 user: [email protected] tags: main, trunk

feat: list a check-in's files in a cone check-in: e90c972c40 user: [email protected] tags: main, trunk

Repo.ListFilesIn passes the public Cone to the enumeration seam as the range predicate (draft-fossil-sparse-cone-00 §4.1). On a tree manifest the walk skips sub-manifests the cone cannot reach (§4.5), so over a sparse store it succeeds where ListFiles fails on an elided node. It reads through the attached content cache, via a new internal manifest.ListFilesFilteredCached.

Tracked in #491. check-in: 7156492179 user: [email protected] tags: main, trunk

ci: run the unit suite under -race; give packages 10m check-in: 612977a2d6 user: [email protected] tags: main, trunk

A Race workflow runs every unit package under the race detector after each merge to main, nightly, and on demand. The full suite passes under -race with no data race reported; it takes several times as long as without, so it does not gate pull requests, which already must be up to date with main to merge. Fossil and zit are not installed there; their interop tests skip, and the Tests workflow runs them.

The unit steps' -timeout is per package binary, and conformance alone takes 130-255s on a hosted runner, so 300s failed it on runner variance. It is now 10m. The dst steps keep 300s. check-in: fdd361b5a9 user: [email protected] tags: main, trunk

fix(sync): export OriginNamer so an embedder's transport keeps a horizon cursor check-in: a52911df25 user: [email protected] tags: main, trunk

Sync and Clone keyed the horizon cursor (draft-fossil-horizon-00 §6.2.6) on an unexported syncOrigin method, which only the HTTP transport could satisfy. Any other Transport kept no cursor and announced its whole holding with igot on every pull.

OriginNamer is an optional interface, discovered by type assertion like FramedTransport, with SyncOrigin() string. The HTTP transport implements it in place of the unexported method, so one assertion serves both. The name must identify the peer stably and carry no credential.

Tracked in #488. check-in: 8d16b7289c user: [email protected] tags: main, trunk

feat: read an artifact by hash and cache content on Repo check-in: 5e882bf1f1 user: [email protected] tags: main, trunk

Repo.ReadBlob expands one artifact by its full hash after the availability check ReadFile applies, and names an unknown hash with ErrArtifactNotFound.

Repo.SetContentCache attaches a byte-bounded content.Cache that ReadFile, ReadFileAt, ReadBlob, ListFiles, Diff and Annotate share; check-in parses are kept in its derived memo, so reads at one check-in parse the manifest once. Default off. A read that finds the DB's finished-write count moved swaps in an empty cache, so a purged, compacted or elided artifact, and a rid SQLite reuses after a purge, is never served from it. Writes by another process are not seen; the method's doc says so.

Tracked in #492. leaf check-in: d367012009 user: [email protected] tags: feat/read-blob-content-cache, trunk

DB.Writes counts pool Exec calls that returned and WithTx transactions that ended. Counting at the end is what lets a reader holding state across calls treat an unchanged count as no write having landed in between. check-in: 51be37dc1a user: [email protected] tags: feat/read-blob-content-cache, trunk

ci: check gofmt; format the tree check-in: 969c8f1fe4 user: [email protected] tags: main, trunk

Sixteen files had drifted from gofmt: struct and comment alignment, one-line if bodies in auth_test.go, trailing blank lines, and a blank comment line gofmt now puts before an indented block in two delta.go doc comments. The test job now fails on any tracked .go file gofmt would change, and lists it. check-in: b2dcbb7130 user: [email protected] tags: main, trunk

ci: pin zit interop to 0.7.3 check-in: 95610ffc23 user: [email protected] tags: main, trunk

zit 0.7.3 (version-0.7.3, check-in f18e3cc041) carries draft check-in 39546186f4, the minted seed floor of horizon §5.2.2 that go-libfossil implements since #485, so the pinned interop zit and the pinned draft copy agree again. Every zit interop test passes against it, none skipped, with REQUIRE_ZIT_BIN and REQUIRE_ZIT_SEED_DATE set. check-in: 10737ceaec user: [email protected] tags: main, trunk

feat(commit): carry files by UUID; fix Create's default login check-in: d0abc1b3fa user: [email protected] tags: main, trunk

A carried file outside a sparse store's cone skips the held check, so any caller text of 40 bytes or more reached the F card verbatim, including a newline that writes a card of its own into the check-in. checkCarried now refuses a UUID that is not a lowercase SHA1 or SHA3-256 hash, which covers FileToCommit.UUID and the sparse checkout path alike (tracked in #486).

The default login follows fossil's db_create_default_users, which checks FOSSIL_USER before USER, LOGNAME and USERNAME (tracked in #487). leaf check-in: 2237a3781a user: [email protected] tags: feat/carried-uuid-default-login, main, trunk

Create with an empty CreateOpts.User read only $USER and then panicked in db.SeedUser on the empty login whenever that variable was unset, as it is in many containers and CI runners. The default login is now chosen as fossil(1)'s db_initial_setup chooses it: the first of USER, LOGNAME and USERNAME that is set, else "root". check-in: c15a6c60e9 user: [email protected] tags: feat/carried-uuid-default-login, main, trunk

FileToCommit gains a UUID field. When set, the F card names that artifact verbatim and nothing is read or stored, reaching the internal manifest.File.UUID path that sparse checkouts already use. A caller that lists a parent's files can now commit with PartialManifest, dropping files, by naming the unchanged part of the tree instead of re-reading and resubmitting every byte of it. Both UUID and Content on one file is refused; so is a UUID the store does not hold outside an eliding cone. check-in: 37bba3f7b9 user: [email protected] tags: feat/carried-uuid-default-login, main, trunk

fix(horizon): date a seed no earlier than the origin's minted floor check-in: d47958493b user: [email protected] tags: main, trunk

A mint wrote the floor on every structural pull and failed the request when it could not, so a read-only mirror or a store whose write lock was held no longer served the structural horizon. The floor is now written only when D moves later, and a failed write is ignored: the mark is a SHOULD, and without it the next seed is dated from the tag rows as before. A stored value that is not a canonical integer, which an overflowing cast would rank above every D, is replaced. check-in: 8356d3d19e user: [email protected] tags: feat/horizon-seed-floor, trunk

Shunning and purging the latest tag event removed the tagxref rows a seed's D was dated from, so the next seed was earlier than the one every structural clone held. Each clone's D floor refused it as a replay and it fell back to the content horizon until the repository's tags caught up with the purged tag's time; a far-future tag held it there indefinitely.

The draft (zit 39546186f4, §5.2.2) now has an origin date a seed no earlier than the latest D it has minted and keep that D across restarts. Mint reads the mark from the store's horizon-seed-floor config row, dates the seed at the later of it and the latest tag event, and raises the row after minting with a conditional upsert, so concurrent mints never lower it. No config group transfers the row, so it is never sent.

Re-pins draft-fossil-horizon-00 at 39546186f4 and adds the conformance case horizon.5.2.2.seed-date-shun, which fails without the floor. check-in: 1fd0dfdbc1 user: [email protected] tags: feat/horizon-seed-floor, trunk

chore(release): v0.16.1 check-in: 4ac6baf2ea user: [email protected] tags: main, trunk, v0.16.1

feat(horizon): re-pin to zit caeef421ee: repository-wide seed D and the receive rule check-in: 7b6f43f1dc user: [email protected] tags: main, trunk

zit 0.7.2 (tag version-0.7.2, check-in 1dcb2d4a29) dates a seed at the repository's latest tag event, as go-libfossil now does. Its horizon draft is the caeef421ee text pinned here (artifact ab18fa7f89).

- ZIT_VERSION is 0.7.2, and ZIT_SHA256 is the checksum of zit-0.7.2-linux-x86_64.tar.gz, checked against the release's SHA256SUMS. - REQUIRE_ZIT_SEED_DATE=1 sits next to REQUIRE_ZIT_BIN, so TestHorizonSeedDateZit cannot skip in CI. - The README, horizons.md, CHANGELOG and test comments now say interop is pinned to zit 0.7.2. check-in: 0c30e0d32e user: [email protected] tags: feat/horizon-dfloor-receive, main, trunk

TestHorizonSeedDateZit has zit mint the seed for a repository whose newest tag event is a cancel, and requires it to match go-libfossil's byte for byte. Both now date a seed at the repository's latest tag event (§5.2.2).

No string in the binary tells a zit from before caeef421ee apart: its version still reads 0.7.1. So the test recognises an older zit by an earlier D in an otherwise identical seed, and skips it unless REQUIRE_ZIT_SEED_DATE=1. CI's pinned zit 0.7.1 skips. A zit built at caeef421ee passes with the variable set. leaf check-in: ad82b4a46d user: [email protected] tags: feat/horizon-dfloor-receive, main, trunk

Re-pins draft-fossil-horizon-00 to zit trunk check-in caeef421ee (file artifact ab18fa7f89), which changes §5.2.2, §5.2.3 and §6.2.3 and adds two Appendix C bullets. This brings in the local feat/seed-receive-drop branch, which implements the receive rule, and resolves it against main (#477 chain-walk bounds, #478 seed binding).

§5.2.2, the seed's D (tracked in #479): - Mint dates a seed at max(tagxref.mtime) over every row in the repository, cancels and propagated rows included, as zit's latestTagEvent does. It was the latest of the rows the seed summarizes. - A cancel of the newest tag, or a reopened closed branch, therefore no longer mints a seed earlier than the one a clone holds, which the clone's D floor refused as a replay.

§5.2.3, the receive rule: - A pull, clone or sync drops a seed by this store's label or by a binding card when it can read it, and the link step refuses one completed later. - The draft also requires that a delta kept without its base, pushed or received, is not announced or served while it cannot be expanded. feat/seed-receive-drop only withheld pushed ones. The pending mark moves into storeDeltaContent, so a pull marks too, and a clone marks what is still unreadable when it ends (seedcheck.WithholdUnreadable). That avoids judging every dangling file delta of a clone a second time. - Since #478 minted seeds carry a binding card. The seed-shaped-ordinary cases therefore strip it, and a new test checks that a seed with the card is never linked.

Conformance rows: - HZ-5.2-2 takes the new D text. - HZ-5.2-4 loses the old ordinary-artifact sentence. - HZ-6.2-6 says the D is fixed by the repository's tag events. - New rows: HZ-5.2-14 (the receive-path drop; PARTIAL, since a seed completed in a later round is kept inert, and was counted when its delta arrived), HZ-5.2-15 (the dropped-hash record) and HZ-5.2-16 (withholding unexpandable deltas). - New cases: two D cases mirroring zit's "horizon seed date" regression, a cancel on C and a cancel at an ancestor below it, and five receive cases. - The deepen-floor case now replays an older seed, since an honest origin no longer mints one. check-in: 5d0ef566aa user: [email protected] tags: feat/horizon-dfloor-receive, main, trunk

chore(release): v0.16.0 check-in: 2f1e5f1e3d user: [email protected] tags: main, trunk, v0.16.0

ci: install pinned zit 0.7.1 so the zit interop tests run check-in: ff2929c770 user: [email protected] tags: main, trunk

The zit interop and horizon-seed tests compared bytes with a real zit only on machines that had one; CI installed none, so every one of them skipped there. The modernc job now installs zit's released 0.7.1 Linux tarball, pinned by its SHA256SUMS entry as fossil is, and runs the unit tests with REQUIRE_ZIT_BIN=1 so a missing or outdated zit fails the run. check-in: 4f2abff4ec user: [email protected] tags: main, trunk

feat(horizon): bind each seed to its horizon and sign seeds apart check-in: b35d577a54 user: [email protected] tags: main, trunk

Every seed an origin mints now carries T +horizon-seed <C> <rule> (draft-fossil-horizon-00 §5.2.1), inside the signed bytes. The card is not counted as a tag in force, Mint never derives a horizon-seed row into a seed, and Apply skips the card.

Before a clone holds a seed, signed or not, it checks the seed against its store cone (§5.2.4). The seed must carry exactly one binding card naming the store's C and rule, and its D must be no earlier than the held seed's. A seed that fails is refused as a wrongly signed one and the clone falls back to the content horizon. A deepen discards the held seed first, so the floor restarts at the new C. A held seed that is no longer bound is discarded when it is next applied.

Seeds are signed with their own horizon-seed-command and are no longer signed by clearsign and pgp-command. SSH seed signatures use the fossil-horizon-seed namespace. clearsign.Key.Verify now takes the namespace, and check-in verification keeps fossilscm.

Adds 9 conformance cases and flips HZ-5.2-8, HZ-5.2-11, HZ-5.2-12 and HZ-5.2-13 to covered. The zit interop gate moves to zit 0.7.1, whose seeds are byte-identical with ours. check-in: 451d368554 user: [email protected] tags: main, trunk

fix(sync): bound basis-chain walks at the read depth check-in: 8d3b1ef28a user: [email protected] tags: main, trunk

Any pusher could hold the write lock about 30s per push with a two-delta cycle. The receive pass's basis-chain walks, and the horizon seed check's pending-row walk, ran to 1<<24 steps before giving up.

All of them now stop at content.MaxDeltaChainDepth, the depth at which content.Expand stops, so no walk outlasts what a read could ever use. What crossing the bound means at each site:

- chainEnd (rootOf, for a dangling delta being stored) keeps a visited set. A chain that loops is refused, as it was, but at the revisit rather than after 1<<24 steps. A chain longer than the bound returns errChainTooDeep. storeDeltaContent then stores the delta untracked: the push succeeds, as a push onto a chain between 16k and 16M deep always did, and since no basis can make the delta readable there is nothing to check against its name later. Storing matches the current behaviour, and an honest transfer never fails for a deep chain it did not build. - verifyFrom's cascade carries each row's depth from the basis that arrived, its own chain included. It does not check, or descend past, a row deeper than the bound: no read can produce those bytes, so the pass owes them no check. A delta that does not hash to its name past the bound no longer aborts the pass. - pathRows loses its step counter. Its visited marks already stop it at a row on a path, so it visits each row once, and the cascade's depth check bounds what it verifies. - seedcheck's chainRoot gives a chain that loops or runs past the bound no root, consistent with ErrChainUnresolvable: the row stays pending, withheld, and is never judged.

Tests: - Closing a two-delta cycle is refused in under 5s (it took 27s). - A push onto a chain one past the bound is stored. - chainEnd stops exactly at its limit. - A dangling delta with a wrong name one node past the bound does not abort the push, whether the basis that arrives is full text or a delta deep on a held chain, while one within the bound is still checked and readable. - MarkPending gives an over-deep chain no root, so filling its far end examines nothing, and returns at once on a cycle. - TestHorizonOverDeepPendingChainDoesNotBlockFill still passes. - Removing each bound, the cycle check, the untracked store or the root depth fails a test. - chainDepth's own early stop is an equivalent mutant: the cascade's root has just been expanded, so its chain is within the bound unless the store is damaged. check-in: eac16cf7ed user: [email protected] tags: main, trunk

feat(horizon): named horizons with depth, and origin seed refusal check-in: 5a7c2ba12e user: [email protected] tags: main, trunk

Review of a067c6e. JudgeFilled treated content.Expand's chain-bound error as a store fault and returned it, and DeltaStored failed storeDeltaContent with it.

A pusher could plant a dangling chain longer than maxDeltaChainDepth under a basis R, building it by re-rooting. After that, every honest push, pull or clone that supplied R as a delta failed for good.

Fix: - content now wraps its cycle and depth errors in ErrChainUnresolvable, and exports MaxDeltaChainDepth. - seedcheck counts ErrChainUnresolvable as unreadable, alongside a hash mismatch and a delta that does not apply or fails its checksum. Such a row is skipped and stays pending, so it stays withheld. - Real faults are still returned, such as content that does not decompress. - The classification uses errors.Is on the sentinel, so the expansion JudgeFilled already performs is not repeated.

Tests: - TestHorizonOverDeepPendingChainDoesNotBlockFill plants a pending row one node past the bound under R, then pushes R as a delta on a held basis. The push succeeds, R is stored and readable, and the over-deep row stays withheld. - TestUnresolvableChainSentinel pins the sentinel for a cycle and for a chain past the bound. - Dropping the classification, or either wrap, fails a test.

A cycle never reaches seedcheck: the receive pass's own basis-chain walk (receive_pass.go chainEnd) refuses it first. check-in: 8b946de415 user: [email protected] tags: main, trunk

Two review nits.

Liveness. A pushed artifact pending on a basis P stayed withheld for good when a pull supplied P as a delta: - Only MarkPending re-rooted, and only for a pushed delta. - Filling a phantom with a delta runs no cascade, so nothing judged the waiting rows if the delta was readable, or moved them to P's own root if it was not. A later fill of that root never reached them.

The fix is one hook at the choke point every received delta passes, storeDeltaContent: seedcheck.DeltaStored. - If the stored delta is readable, the rows pending on it are judged. - If not, they move to its chain root, so whatever finally resolves the chain judges them. - It replaces MarkPending's push-only re-root.

Error surfacing. JudgeFilled still skips a row whose own bytes cannot be read (a delta that does not apply or fails its checksum, or bytes that are not the artifact named) and judges the rest of the batch. Any other failure is now returned, a broken chain to the filled root included: check-in and tag return it, and the cascade logs it, as it already did for JudgeFilled errors. The separate availability skip went with it.

Tests: - A pushed plain file and control artifact pending on a basis that a pull supplies as a delta, on a held basis or on one that arrives in the next round. Both become readable and stop being withheld, are announced on a push-on, and the control artifact is linked. - A row whose content does not decompress is reported. - A checksum-failing row is skipped like an invalid one. - Each hook branch and each error class fails a test when changed. check-in: 584c55d1fd user: [email protected] tags: main, trunk

Pin draft-fossil-horizon-00 at zit 236b7b4285 (artifact 130b85e386), which adds named horizons (§6.2.7), the origin's own seed label and the refusal of a pushed seed on expanded content (§5.2.2, §5.2.3, §5.7), and the seed's horizon-seed binding card with the clone's checks on it.

Re-read horizon.tsv against the new text: the seed-label, push-check and named-horizon rows follow its wording; the client rows of named horizons move from §6.3 to §6.2.7 (HZ-6.2-19 to HZ-6.2-21); the store-local judging of a late seed and the binding-card shape refusal get rows of their own (HZ-5.2-9, HZ-5.2-10). The binding card, its no-derive rule, the clone's binding and D-floor checks and the separate seed-signing namespace (HZ-5.2-8, HZ-5.2-11 to HZ-5.2-13) are not built and show as gaps. check-in: cfa0577b2a user: [email protected] tags: main, trunk

# Conflicts: # conformance/horizon_test.go # docs/site/data/conformance.json check-in: 81d00b2e3b user: [email protected] tags: main, trunk

Takes the client-side announcement fix. A seed refused at the link step without a pending row (one a pull or clone completes in a later round) leaves unclustered through refuse(). A push-on from that store never names it, and a test now pins this. check-in: 9f0b995575 user: [email protected] tags: feat/horizon-dfloor-receive, main, trunk

Review of 0e28b6a: when the origin pushed on to a peer, its own client announced artifacts its replies withhold. A pending seed went out from unclustered, and after verify.Rebuild (which marks every blob unclustered and unsent again) a refused seed went out too. The peer kept asking, loadFileCard withheld it, and the push ran out of rounds.

Fixed in the tables the client announces from, not in each reader: - MarkPending takes a pending row out of unclustered. - decide puts it back once judged not a seed; a seed stays out. - rebuildBookkeeping leaves withheld rids out of unclustered and unsent. - That also keeps them out of cluster minting, which reads unclustered. - sendPrivate and the private gimme answer, which read the private table, skip withheld rids. - sendUnclustered needs no filter of its own, and sendAllClusters lists only linked clusters, which a withheld row never is.

JudgeFilled now skips a row that does not expand (it stays pending) and judges the rest of the batch, instead of returning.

Tests: - A pending seed is not announced on a push-on, publicly or privately. - A peer asking for a private pending seed outside a private sync is not told it exists. - After a rebuild, a push-on converges and never names the refused seed. - A pending file, once judged, is announced again on a push-on. - An unreadable row does not stop the next fill in the batch being judged. - Every new mechanism fails a test when disabled. check-in: df78b8aee9 user: [email protected] tags: main, trunk