BranchLeaves joined the leaf table without excluding leaves that carry a closed tag, so a closed leaf still counted toward a branch's leaves, and WouldFork, built on it, reported a fork for a version that was not itself a leaf. This made a closed leaf look like an open one and a non-leaf commit look like it would fork.

BranchLeaves now excludes any leaf with a closed tag (tagtype>0), matching fossil's leaf_is_closed_sql, so it returns a branch's open leaves. WouldFork is now fossil's leaf_ambiguity: the current version must itself be an open leaf (is_a_leaf AND not closed) before another open leaf on the same branch can make a commit fork; a non-leaf or closed version returns false. currentBranch now surfaces a real query error instead of masking it as "trunk". Unit tests cover the open-sibling, closed-sibling and non-leaf-parent cases. check-in: f8496d12cc user: [email protected] tags: main, trunk

The new erratum to draft-fossil-sync-protocol-00 §7.3 has the server answer uvgimme only for a requester with Read, with Clone implying Read, and refuse silently otherwise. fossil 2.28 answers any uvgimme.

- The erratum is copied unmodified into conformance/rfcs/erratum and pinned in requirements/sync.tsv, with two new requirements, SP-7.3-11 and SP-7.3-12. - Server cases can carry a fossil note again: a case that expects something fossil(1) does not do says why, and fails once fossil(1) does it. sync.7.3.uvgimme-no-read uses it, so the case flags the day CI's pinned fossil takes the matching patch. - sync.7.3.uvgimme-clone-implies-read finds that go-libfossil's read gates ignore Clone; it is marked as a gap, tracked in #312. check-in: 2cb62c37bc user: [email protected] tags: main, trunk

* test(conformance): sync-protocol conformance suite (#262)

Checks go-libfossil against draft-fossil-sync-protocol-00, requirement by requirement: 149 requirements, 199 cases. Server cases send raw requests to Repo.XferHandler and, through `fossil http --in`, to fossil(1) on a copy of the same repository; client cases drive the public Sync and Clone against a scripted peer, with fossil(1)'s client run against the same script for most of them; HTTP cases cover redirects, status codes and media types; interop cases run whole exchanges in both directions.

Known failures are marked with the placeholder #0 until their issues are filed, so TestCoverage fails until then.

* test(conformance): sync suite review fixes (#262)

Cases for the requirements that had none where fossil(1) can show the behavior: server-version, the SHA3 gate, shun admission, cookie and cleartext logins, pushed configuration, ci-unlock, cluster brackets, the clone's unversioned catalog and the pull-only message. Two rows added: link ARG handling and the short uvigot.

A fossil(1) disagreement, a harness failure or a deviation other than the one named now fails a case whatever its mark. The server's log output is discarded in the tests.

* test(conformance): mark sync gaps with their issues; drop gaps v0.11.1 closed (#262)

v0.11.1's read gate and redirect policy make seven sync cases conform, so their gap marks are removed. The rest are marked with the issues that track each fix: #296 and #299-#310.

* test(conformance): accept either fossil refusal of a negative clone SEQNO

fossil 2.28's macOS build answers clone 3 -1 with clone_seqno 0 and its Linux build, which CI pins, with the invalid-sequence error alone. The draft leaves the reply open; the case now holds go-libfossil only to sending no content. check-in: 80c72b27c7 user: [email protected] tags: main, trunk

* fix(sync): send content only to requesters allowed to read it

The sync server answered `gimme HASH` with the artifact whether or not a pull card had been accepted, and answered `pragma uv-hash`, `uvgimme` and `uvigot` with the unversioned catalog and files for any requester, anonymous ones included.

Match fossil's page_xfer:

- gimme is answered only when a pull card was accepted, or in a clone, which fossil also treats as pulling. - pragma uv-hash sends the catalog only to a requester with Read (o, i or Admin). uv-push-ok goes only to a requester with y; other readers get uv-pull-only, as fossil sends. - uvgimme and uvigot send nothing to a requester without Read, and uvigot adopts a client's mtime only for a requester with y. - Table sync follows the same rules: xgimme needs an accepted pull and pragma xtable-hash needs Read. - pragma ci-lock takes or reports a lock only for a requester with Write, so an anonymous requester cannot learn who holds one.

Pragma handling moves into handlePragma to keep handleControlCard short.

GHSA-mvx3-6793-8x4r

* docs(changelog): note that gating ci-lock on Write departs from fossil

GHSA-mvx3-6793-8x4r

* fix(sync): follow HTTP redirects as fossil does, never HTTPS to HTTP

Both HTTP transports used net/http's default redirect policy. It follows a redirect from HTTPS to plain HTTP, so a 307 or 308 resent the request, login card included, in cleartext. It also turns a 301 or 302 into a GET without the body and stops after 10 redirects, neither of which fossil does.

PostXfer now follows redirects the way fossil's http_exchange (src/http.c) and draft §4 describe. It replaces the client's redirect policy for these requests, then:

- refuses a redirect from HTTPS to HTTP, and to any scheme but http(s); - re-POSTs the same body and Content-Type for 301, 302, 307 and 308; - strips one trailing /xfer from the new location, as fossil's loop does (the draft says "repeatedly"; the code strips once); - fails on the twentieth redirect (MAX_REDIRECTS 20).

HTTPTransport and NewHTTPTransport both use it, including with a client passed through WithHTTPClient.

GHSA-3f8x-8fr5-mr98 check-in: 3a1014cf60 user: [email protected] tags: trunk

* test(conformance): repository-state conformance suite (#262)

Check go-libfossil against draft-fossil-repo-state-00, requirement by requirement. Each derived-state case is a scripted sequence of artifacts loaded into go-libfossil and into fossil(1); the tables derived from it are compared row by row, joined by artifact hash. Every case runs through manifest.Crosslink, as one sweep and one artifact at a time, and most also run as a ".rebuild" case through verify.Rebuild. fossil's rows must equal each expectation on every run, and each case states what ignoring its requirement would change, so a case cannot pass or fail by accident.

API cases cover name resolution, leaves, the ambiguity test, branch creation and listing, and tagging, each compared with fossil's answer on the same repository. TestRepoStateCorpus compares verify.Rebuild and a Crosslink sweep with fossil rebuild on real repositories named by CONFORMANCE_REPOS.

Known failures are marked with the placeholder Gap #0 until their issues are filed, so TestCoverage refuses them for now.

* test(conformance): repository-state suite review fixes (#262)

- Unsorted T-card cases use a non-hex name, so the ordering rule, not the hex-name rule, is what rejects them. - RS-4.2-7 crosslinks every artifact a second time over the rows the first left (repostate.4.2.relink-unchanged), instead of a sweep that links nothing. - RS-7.3-2, -3 and -4 are checked through manifest.Checkin and branch.Create, which maintain leaves one check-in at a time; a crosslink sweep recomputes the whole table. - retag-branch is built so the retagged check-in's own leaf status changes; branch membership is read through BranchLeaves, and the tagxref case it replaced is reattributed to RS-5.4-5; bad-prefix also compares tagxref. - parentless-untagged follows the draft and marks go-libfossil's crosslink, which follows fossil(1), as a deviation; the unused draft field becomes fossil, checked against fossil's rows. - Every fossil(1) command, in all suites, runs with HOME, FOSSIL_HOME and XDG_CONFIG_HOME in the test's directory, so it no longer records checkouts in the user's global configuration.

* test(conformance): mark repository-state gaps with their issues (#262)

Replace the placeholder gap marks with the issues that track each fix: #278 and #285-#294. check-in: 7fa0b9b5da user: [email protected] tags: trunk

* fix(xfer): reject non-decimal card sizes and bound payload memory

A payload card's size was parsed with strconv.Atoi, so a peer sending `file HASH -1` reached readPayload's negative-size assertion and panicked the client or server decoding the message, and a signed size such as `+5` was accepted. Sizes now go through parseSize, which follows the sync draft's `decimal = 1*DIGIT` grammar and fossil's 32-bit limit and returns a decoding error for anything else. uvfile FLAGS use the same parser, so negative FLAGS are rejected as the draft requires. The assertion stays, now unreachable from peer input.

readPayload no longer allocates the declared size up front: its buffer grows with the bytes read, so a large size over a short message fails after allocating about what was sent.

A cfile payload may inflate only to its USIZE (a margin above it for a delta) and not past its 4-byte size prefix, and all cfile payloads in one message share a MaxCFileInflateBytes (1 GiB) budget, so a few megabytes of zlib cannot inflate to tens of gigabytes.

FuzzDecode exercises the decoder over both framings.

* fix(sync): refuse malformed xrow payloads and cap HTTP reply bodies

An xrow payload or xdelete PKData from a peer was decoded into a map and used as is. JSON null gave a nil map that reached coerceJSONNumbers' assertion, and a primary-key value of the wrong type or a nested value reached normalizeValue's, panicking the client or server. Both directions now decode through decodeRowJSON: one JSON object, nothing after it, and repo.ValidateRow's rules, which require scalar values and primary-key values that PKHash can normalize. A client refuses an xrow for a table it has not registered. Column types match case-insensitively, as table creation accepts them.

Both HTTP transports read replies through xfer.ReadBody, which stops at MaxBodyBytes (MaxDecompressedBytes plus framing headroom) and returns ErrBodyTooLarge, untagged for retry, instead of reading an endless body into memory.

FuzzTableSyncPayload drives the server handler and client over arbitrary payloads.

* fix(sync): check synced-table rows and identifiers before building SQL

repo.UpsertXRow wrote a row's map keys into its INSERT as column names, and the rows of an xrow payload or xdelete PKData come from the peer, so a key such as "val, mtime) VALUES(...) --" chose the statement's columns and values, and a key could select from other tables into a synced one. This held on the server for pushed rows and tombstones, and on the client for a server's rows.

A peer's row must now use exactly the table's declared columns: repo.ValidateRow refuses any other key, which also keeps a peer from setting _owner. UpsertXRow no longer trusts its caller: it reads the table's registered schema and refuses any key it does not declare, apart from _owner on an ownership-mode table. ListXRows, LookupXRowOwner and DeleteXRowByPKHash validate the table name and the column names they interpolate, so no synced-table identifier reaches SQL text unchecked.

* fix(sync): answer reqconfig only with fossil's transferable config

The sync server answered reqconfig NAME with any row of the config table, for any requester, including one with no capabilities, so project-code, server-code and every local setting could be read by anyone who could reach the sync endpoint.

handleReqConfig now follows fossil's page_xfer and configure.c. It answers only a /GROUP name, only for a requester with Read, matches the group by prefix in fossil's table order, and sends only the names fossil transfers in that group, in fossil's encoding. The user and subscriber tables also need Admin, which counts only for a verified login, and the concealed table needs RdAddr. A refused request gets no cards and no error.

The groups, their names and their capability rules live in one place, config_groups.go. A differential test answers the same requests with fossil's own server and with HandleSync on the same repository and requires identical config cards.

* test(sync): keep fossil http from leaving a backoffice process

* fix(sync): check the login nonce against the request body

The server verified a login card's signature over its nonce but never checked that the nonce was the SHA-1 of the request body after the login line, as fossil's check_tail_hash does (draft-fossil-sync-protocol-00 §6.2). A login card captured from one request therefore authenticated any other request body.

The decoder now records, for each login card, the SHA-1 of the decompressed bytes after the card's line, and VerifyLogin refuses a card whose nonce does not match it. A login card that was never decoded has no tail and is refused.

Login handling otherwise follows fossil's page_xfer and check_login: - a refused login ends the exchange with "error login failed" instead of continuing as nobody; - a second login card is refused with "multiple login cards"; - a successful login adds the user's capabilities to nobody's instead of replacing them; - nobody and anonymous log in without a password, and the reserved developer and reader logins and accounts with an empty password cannot be logged into.

Fixes GHSA-2g5p-q5f3-477p.

* fix(repo): seed nobody with fossil's default capabilities

A new repository seeded the nobody user with "cghijknorswy", so every request without a login card could push check-ins and unversioned files, write wiki pages and tickets, and held Setup. Seed the four built-in users `fossil new` creates, with fossil's capabilities: nobody "gjorz", anonymous "hz", developer "ei" and reader "kptw".

Setup and Admin now include check-in, read and clone, as they do in fossil, so the creating user can sync once given a password. A login to an account with no stored password is refused, as fossil's check_login refuses it; the creator starts with none, and the signature would otherwise be the SHA-1 of the nonce alone.

Tests that push without credentials grant nobody push explicitly. Existing repositories keep their stored capabilities.

* test(auth): reserved-login test updates the seeded developer and reader rows

New repositories now seed developer and reader as fossil does, so the test gives the existing rows a password instead of inserting them.

* docs(changelog): group the sync hardening fixes under Security

* test(sync): reqconfig tests sign their logins over the request body

The login nonce now has to cover the request body, so the reqconfig tests build their admin requests with signedRequest instead of a bare login card. check-in: 3a6126c3e9 user: [email protected] tags: trunk

The server answered a pull-only client's igot cards with gimme for artifacts it lacked. The client was not pushing, so it never sent them and kept going round after round until it hit the round cap. Fossil's page_xfer handles igot only when the client sent a push card, and does the same here: a pull-only igot just records what the client has.

The client also treated the server's gimme cards as outstanding work even when it was not pushing, so any server that did ask (an older libfossil, or another implementation) could hold a pull open. A session without push now ignores gimme cards.

Two handler tests relied on a pull card alone producing a gimme; they now send a push card. check-in: 1e3e2df2f2 user: [email protected] tags: trunk

* test(conformance): delta-format conformance suite (#262)

Check internal/delta against draft-fossil-delta-format-00, requirement by requirement: 39 requirements, 88 cases. Decoder cases (Apply and OutputSize) are byte vectors, each also given to fossil(1) through its SQL delta functions; fossil's verdict must match unless the case records why it follows the draft instead (fossil masks the high bit, does not verify the checksum, and returns the size as a signed int). Each rejection breaks one rule and keeps the trailer valid, so check order cannot hide the rule under test. Producer cases check Create against a strict oracle and against fossil's delta_apply, and fossil's delta_create against Apply.

FuzzDeltaApply compares Apply with an independent model decoder; FuzzDeltaRoundTrip checks Apply(Create(s, t)) == t. DF-3.1-6 (Create has no 2^32 bound) has no case: it needs a 4 GiB input.

* test(conformance): delta suite names the reason each rejection must give

A rejection case could pass because some other rule rejected the input. Every rejection case now names a substring of go-libfossil's error (or ErrChecksum), and the harness fails the case when the message differs.

Vectors that were rejected for the wrong reason are fixed or reattributed: copy-truncated (DF-4-2), reject-10-nul (non-empty count, so the NUL is the opcode), copy-overflow-64 (DF-4-11), and high-bit-count, which now tells a masking decoder from a conforming one (fossil accepts it). The phantom-base case uses a literal-only delta; the content-discard case checks ErrChecksum and the store-side header refusals move to DF-4.1-3. The allocation bound gets its own row (DF-6-5) and takes the least of five runs. DF-3.1-6 notes issue #279. Drops a duplicate size-query case, the sizeCase helper and the decorative fuzz guard bytes. check-in: 0629de2beb user: [email protected] tags: trunk

* test(conformance): artifact-format conformance suite (#263)

Adds the suite for draft-fossil-artifact-format-00 with errata 4.1.2, 4.2 and 4.6.4, so the parser, writer, naming, hash policy and delta manifest code are checked requirement by requirement instead of by scattered unit tests that each encode one reading of fossil.

- requirements/artifact.tsv: 101 requirements from Sections 3 to 8, pinned to the draft and the three errata. - Parse cases are a known-good base artifact per type plus one edit, with the Z card recomputed, so the edit alone explains a rejection. Every case also runs through fossil test-parse-manifest; fossil 2.28 agrees with all of them. Card legality, duplicates, arity and hash positions are generated from the bases and the draft's own tables. - Accepted inputs must re-encode byte for byte unless marked non-canonical. - API cases cover naming, verification, the comparator, the writer, the R card, the hash policy and delta-manifest enumeration. - FuzzArtifactRoundTrip checks parse/marshal/parse stability, and TestArtifactCorpus (CONFORMANCE_REPOS) parses every artifact fossil crosslinked in real repositories.

267 cases go-libfossil fails are marked with a "#0" placeholder in artifact_gaps_test.go, grouped by the code at fault, for issues to be filed against.

* test(conformance): tighten the artifact suite after review (#263)

A rejection only counts when the rule the case names makes it, so every rejection case that is not about the Z card must now pass deck.VerifyZ first. Inputs go-libfossil rejected only through the Z check (framing, residue, sterilizer, unstripped wrappers, an extra Z token) become gaps, along with VerifyZ itself, which rejects an absent Z card and reads the last byte where Section 6.2 says it must not.

- Cases can name the substring fossil's rejection reason must contain; the unknown-letter cases sit in sorted position, the sterilizer cases are filed under card order, which is what fossil rejects them for, and generated legality cases take their requirement from the draft's table. - The ratchet that never persists the hash policy is a gap: fossil opening the repository still reads auto. - New rows for 4.7.9, 6.2 (absent Z, last byte), 6.3.4 and 7.1. - The audit refuses the #0 placeholder, a test keeps the gap map and the fuzz test's forgiven losses tied to live gap cases, and one Z-card helper replaces two.

* test(conformance): mark artifact gaps with their issues (#263)

Replace the placeholder gap marks with the issues that track each fix (#265-#275), and keep coverage_test.go within 100 columns. check-in: d52a6fe050 user: [email protected] tags: trunk

Adds the conformance package that the per-draft suites build on. Nothing in the repository has yet checked go-libfossil against the Fossil drafts requirement by requirement.

- A Case names its requirement. A Gap (an issue) or Deviation (a reason) marks a known difference. A marked case still runs: while it fails it is skipped, and once it conforms it fails until the mark is removed, so the list of known failures cannot go stale. - Requirements tables pin the SHA-256 of the drafts and errata they were written against. Those are copied unmodified into conformance/rfcs by scripts/refresh-conformance-rfcs.sh. A changed draft fails the coverage test until its table has been re-read and its pin moved. - Coverage is derived from the cases rather than recorded: - COVERED: every case conforms. - PARTIAL: the table's row says what its cases leave out. - DEVIATION: a case is marked as a deliberate difference. - FAILING: a case is marked with an open gap. - GAP: no case yet.

CONFORMANCE_REPORT=<path> writes the requirement-by-requirement report as Markdown. - fossil(1) is run on every case's input, through test-parse-manifest, so a case's expectation is checked against fossil each time the suite runs. withZ appends the Z card, so a case that edits one card never also breaks the checksum.

The harness's own rules are tested in harness_self_test.go. No suites are registered yet; the artifact-format suite is next (#263). check-in: 57a558fcbf user: [email protected] tags: trunk

* fix(manifest): store check-in, link and tag rows as fossil does (#258)

A rebuilt or locally written repository differed from fossil's in three derived values: event and plink times in their last digits, event.omtime left NULL, and a valueless tag stored as '' instead of NULL.

Fossil stores a tag time as SQLite's julianday() bound as a double, but writes event, plink, attachment and forumpost times into SQL text with its own printf("%.17g"), which emits 16 digits in long double arithmetic. TimeToJulian is now julianday() exactly (whole milliseconds over 86400000) and TimeToTextJulian ports fossil's printf conversion, so both match fossil bit for bit. The conversion follows builds where long double is a double (macOS, Windows); fossil itself differs across platforms in the last digit.

Check-in events record omtime, tag writes bind NULL for an empty value, the readers of event and plink times under the ncruces driver rebuild the text form, and the CLI's tag listing reads NULL values.

* test(db): sweep both stored julian forms; scan the text form from driver times (#258)

* test(verify): compare text times bit for bit only where fossil computes them in double (#258) check-in: 2c2e5e74b6 user: [email protected] tags: trunk

* fix(sync): clone and sync exchanges follow the sync draft (#255)

* docs(changelog): clone and sync exchanges follow the sync draft (#255)

* fix(sync): fail unfinished clones; ask for held phantoms once (#255 review) check-in: e46a6c3f60 user: [email protected] tags: trunk

Annotate aligned each version with its parent using its own LCS, so where a line repeats (blank lines, braces) it could pick a different copy as the new one than fossil does, crediting the line to another check-in. It also trimmed every trailing newline, dropping trailing blank lines.

Annotate now uses a port of fossil's line diff (break_into_lines, diff_all, longestCommonSequence and the rest of diff.c that annotation uses), hash included, since the hash orders the candidates and so decides ties. Like fossil's annotate, it diffs each older version against the starting one and credits a line to the version just newer than the first ancestor that lacks it. This also drops the table the size of both versions that the old LCS allocated at every step. check-in: ada7aadbb5 user: [email protected] tags: trunk

Past the first parent, walkParentChain wrote each line's attribution by its index in an older version, so lines were credited to the wrong check-in and an older version longer than the start panicked. Carry each compared line's start-version index through the walk and stop following lines that no longer match. check-in: a63adf1dcf user: [email protected] tags: trunk

The CLI's merge marked a conflict with vfile.chnged=5, which fossil reads as "added by integrate", and recorded no merge-in version or vmerge row, so a merge went unseen and the next commit dropped the merge parent.

Merging now lives in the library as Checkout.Merge and records what fossil records: chnged=2 and mrid for a file taken from the merged-in version, chnged=3 for an added file, a per-file vmerge row for a merged file, and vmerge id 0 for the merge. Conflicts use fossil's markers and are found by looking for them (Checkout.Conflicts), so fossil and go-libfossil read each other's merges. Where fossil relies on its undo copy, Merge keeps the file or refuses instead.

Commit reads vmerge for merge parents, and, as fossil's commit does, refuses unresolved conflict markers, empty commits, unknown names and partial commits of a merge. The CLI's merge, conflicts, mark-resolved and ci commands call the library; ci now commits the checkout. check-in: 7ea6fba63f user: [email protected] tags: trunk

The release workflow's last step sent EdgeSync an upstream-bump event with a personal access token. It has failed with 401 on every release since v0.8.1 because the token expired, marking each release run failed after the release itself was published. EdgeSync also stopped accepting the event when the repository was renamed.

EdgeSync now tracks go-libfossil with Dependabot (danmestas/EdgeSync #199), which needs no token here, so the step is removed. A release run now succeeds once the GitHub release exists. check-in: af8e531e68 user: [email protected] tags: trunk

* fix(checkout): revert undoes a pending rename (#242)

Revert-all selected rows by content and removal, so a rename recorded by fossil mv (origname set, chnged 0) was skipped; a renamed and edited file had its committed content written under the new name, leaving the rename pending. A named revert only matched the current name.

Like fossil's revert, a pending rename is now undone: the file under the new name is removed, the row takes its old name back (UPDATE OR REPLACE, as fossil does), and the committed content is written there. Revert-all includes renamed rows, and a named revert answers to either the new or the old name. A renamed file a merge added is still removed outright, as before and as in fossil.

removeRegularFile is shared by the rename undo and removeMergeAdded, so neither deletes a directory the user made at that path.

* fix(checkout): plan a revert before changing anything (#242)

Undoing a rename with UPDATE OR REPLACE let a swap destroy files: reverting a.txt and b.txt exchanged through fossil mv deleted the other row, the loop then removed a file it had just restored from its stale row list, and the next restore failed half way, leaving both files gone from disk. A new file added under the old name was overwritten the same way. A named revert of the old name also stopped at the first match.

Revert now plans, checks, then acts. planRevert decides every target row's action (restore, un-manage, remove a merge-added file, undo a rename) before anything changes; checkRenameUndoFree refuses the whole revert when undoing a rename would land on a name another tracked row holds, since go-libfossil has no undo copy to recover an overwritten file (fossil saves one); revertFile carries out the planned action. The rename undo is a plain UPDATE, which fails rather than replace a row. A name selects every row it matches, new name or old. The callback reports a rename under the name the file gets back.

Swaps and chains, which fossil completes via its undo store, are #248.

* fix(checkout): revert named paths as one; refuse untracked file at old name (#242)

Review of the planned revert found two remaining ways to lose work:

- Several named paths were reverted one at a time, so in `revert b.txt z.txt` the edit to b.txt was discarded before z.txt's rename undo was refused. revertPaths now gathers every named row (new or old name) and plans and checks them as one revert: a refusal reverts none of them. - Undoing a rename restored the committed content over an untracked file at the old name. checkRenameUndoFree now also refuses when a file on disk there differs from the committed content (wouldClobber), for the same reason it refuses a tracked one: there is no undo copy.

* docs(changelog): note revert's untracked-file and multi-path rules (#242) check-in: 50d1a7f72b user: [email protected] tags: trunk

* fix(content): report database faults from the availability check (#239)

IsAvailable and AvailableByUUID return a bool, so a busy, closed or I/O-failed database read as "content missing". Where missing content is tolerated on purpose, that swallowed the fault: since #231, CreateCheckout leaves vfile empty for a partly synced tip, and a fault during resolveFiles' availability check took the same path, so Create succeeded with an empty checkout.

The chain walk moves to checkAvailable, which returns an error for a fault and a plain false for an unknown rid, a phantom, a cycle or an over-long chain. New CheckAvailableByUUID exposes that distinction, and resolveFiles uses it, so a fault fails LoadVFile with its own error instead of errMissingContent. IsAvailable and AvailableByUUID keep their signatures and their fail-closed reading of a fault; their ~30 callers treat "not available" as a failure already.

* docs(changelog): note the availability fault fix (#239)

* fix(content): NULL size is missing content; don't cache faults (#239)

checkAvailable scanned blob.size into an int64, so a NULL size (legal in the schema) became a hard error in resolveFiles where it used to read as missing content. Both walks now read a NULL size as a phantom.

AvailabilityCache memoized a walk's verdict for every node it visited, including a false that came from a database fault, so one transient fault stayed wrong for the rest of the crosslink sweep. A faulted walk now returns false without being remembered; an unknown rid is still a verdict and is cached as before. check-in: 4b14509374 user: [email protected] tags: trunk

* fix(checkout): dry runs change nothing; switching ends a merge (#236)

A dry-run Extract still called LoadVFile(rid, clear=true) and finalizeExtract, replacing the current version's vfile rows (pending adds, renames, merge state) and moving vvar to the target. A dry-run Update still ran finalizeUpdate, with the same effect.

Extract now lists the target's files from its manifest (resolveFiles) instead of reading them back from vfile, so a dry run never loads them; only a real extract replaces vfile and records the version. Update skips finalizeUpdate on a dry run.

finalizeExtract also clears vmerge, as fossil's checkout does: a forced switch used to leave a merge record that would give the next commit a bogus merge parent.

* fix(checkout): forced extract starts fresh; re-extract keeps a merge (#236)

Two gaps against fossil's checkout found in review:

- A forced extract dropped only other versions' vfile rows (LoadVFile keeps the target's), so re-forcing the current version left a pending rename's two names on disk and in vfile, and a removed file restored but still marked deleted. fossil checkout --force deletes every row first; replaceFileList now does the same when Force is set. - Extract cleared vmerge on every completed run. fossil's checkout returns early when the version does not change and Force is off, keeping a pending merge; finalizeExtract now ends the merge only for a switch or a forced extract.

* refactor(checkout): finalizeExtract reads the prior version itself (#236)

Only the final step needs the version being replaced, and nothing changes vvar before it, so finalizeExtract reads it there. A dry run no longer calls Version at all, and Extract drops to 64 lines. check-in: 139d426384 user: [email protected] tags: trunk

* fix(cli): add, rm, rename and revert call the library (#234)

These commands wrote the checkout's vfile with raw SQL, each with its own copy of rules the library already gets right, and ignored most Exec errors. rename recorded the new name but left the file under the old one, so the next commit failed on a "missing" tracked file. revert only cleared change flags over an edited file and restored nothing. add wrote a SHA1 mhash whatever the repo's hash policy.

They now open the checkout through the library (openWorkingCheckout, which finds the repository from the checkout when -R is not given) and call Add, Remove, Rename and Revert, returning every error, including from closing. rename moves the file, like fossil mv --hard; revert restores committed content, like fossil revert.

Two library gaps the CLI exposed are fixed where they live, matching fossil's revert: - Revert refreshes change flags from disk before choosing files, so an edit no scan has recorded yet is reverted instead of skipped. - Reverting a never-committed add un-manages it and leaves the file on disk. It used to delete the file, destroying the only copy; fossil prints UNMANAGE and keeps it.

revertFile's restore path moves to restoreCommitted, bringing it under 70 lines. The merge, merge-resolve and conflicts commands are #244.

* docs(changelog): note the CLI and Revert changes (#234)

* fix(checkout,cli): revert and rename match fossil; find repo without -R (#234)

Review of the CLI switch found the library and the new helper falling short of fossil:

- Without -R, every command failed on a fresh fossil checkout: the repository lookup opened the checkout database read-only, which clashes with the WAL pragma on a database fossil left in delete journal mode, and hid the real error. It now opens it as the library does, reports the error, and resolves a relative repository path, as fossil open stores it, against the checkout's directory. - Rename moved the file onto an existing untracked file, destroying it. It now refuses, as fossil mv --hard does, before changing anything. - Revert skipped a file missing from disk (no change flag records it) while the CLI reported it reverted. A named file is now reverted whatever its flags say, and revert-all includes missing files. - Revert left merge state behind: restored rows kept mrid on the merged-in version and vmerge kept its row, so the next scan saw the file changed and the next commit recorded a merge parent. Restored rows now reset mrid and mhash, and revert-all clears vmerge, as fossil's revert does.

Rename's vfile lookups move to renameRowID and revert-all's selection to needsRevert, keeping both functions under 70 lines. The re-add test now checks the vfile row itself, since fossil's own scan would hide a stray chnged=1.

* fix(checkout): revert removes a file a merge added, as fossil does (#234)

Revert decided between restore and un-manage by rid alone. A file a merge added has a non-zero rid (its merged-in artifact) but is not in the checked-out version, so revert restored it as if committed and left it tracked; the next commit then included it. Fossil looks the file up in the version's manifest and, when it is absent and rid is not 0, deletes the file and its row (its content lives on in the merged-in version).

revertFile now takes the vfile row and the version's path set and does the same: a pending add is un-managed and kept on disk; a file the version lacks, under its name or its name before a pending rename, is removed; anything else gets its committed content back. Checking the pre-rename name keeps a renamed, edited file from being deleted.

Rename's refusal message now fits a directory at the destination too.

* fix(checkout): never remove a non-file when reverting a merge add (#234)

removeMergeAdded deletes only a regular file at the merge-added path; anything else there, such as a directory the user made, is left alone. Revert's doc now says edits to reverted files are discarded: fossil saves an undo copy first, go-libfossil has no undo yet. check-in: de8f3ed302 user: [email protected] tags: trunk

* refactor(vfile): give the checkout's vfile encoding one owner (#233)

Checkin, stash and status each decoded fossil's vfile columns on their own: rid=0 for an add, origname==pathname on unrenamed rows, chnged merge codes. That is how #228 happened, and stash still had it wrong: it selected chnged=1 only, so files changed by a merge (chnged 2..9) were silently left out, and it stashed a rename as a plain edit.

New package internal/vfile owns the encoding. Load reads a version's rows; Row answers what callers ask (IsAdded, IsRemoved, RenamedFrom, ContentChanged, PendingMerge); Classify gives the change type in fossil's order. checkout.FileChange is now an alias of vfile.Change, so the checkout API is unchanged.

internal/checkout's status, Extract safety check and checkin read rows through it. The never-read vfileCommitEntry.changed field is gone.

internal/stash selects changes with vfile.Classify against the checked- out version and refuses a pending merge or rename by name, since its save and revert paths restore files by current name against the committed version and would drop either. Its store-and-revert loop is split into stashRecord and revertStashedFile; removed and modified files share one revert path, keyed by row id.

* fix(vfile): a pending merge is chnged 2-5 or mrid != rid (#233)

PendingMerge counted every chnged above 1 as a merge, but 6 through 9 record a change to the executable or symlink bit; fossil leaves chnged=6 after a plain chmod +x. Stash then refused an ordinary permission change as "a pending merge". It also ignored mrid: an edit made on top of a merge drops chnged back to 1 while mrid still names the merged-in content, so the refusal could be bypassed and the merge half-dropped.

Row now carries mrid; PendingMerge is chnged 2..5 or mrid != rid, and ModeChanged covers 6..9. Stash refuses a permission change with that reason, since its revert writes default permissions.

Load reads a NULL rid as 0 and a NULL mrid as rid, the tolerance the checkin code had when it scanned rid into a NullInt64.

* refactor(checkout): ask vfile.Row about merges in Extract's check (#233)

Extract's safety check relaxed only rows with chnged == 1, decoding the column itself. It now asks the row (not a pending merge, not a mode change), keeping fossil's encoding in internal/vfile. Equivalent for every real row. nextChangedState's comment no longer calls the mode changes (6..9) merge states. check-in: f55d0ed2de user: [email protected] tags: trunk

* fix(checkout): classify changes in one place; report missing files (#232)

HasChanges, VisitChanges and Extract's safety check each decided what counts as a change: two SQL WHERE clauses and classifyChange, with an error path whose only job was catching their disagreement. They had already drifted once (HasChanges missed renames until #229).

Now one reader, loadVFileRows, returns every row of a version and one classifier, classifyChange, judges it, in the order fossil's changes command uses: removed > missing > added > renamed > modified. A renamed file that is also edited stays "renamed", so callers that see only the change type do not lose the rename. The WHERE clauses and the malformed-row error are gone.

VisitChanges with a scan now reports a tracked file that is gone from disk as ChangeMissing, which the public API already exposed as "missing" but nothing produced; such files were left out, or shown as added or modified. HasChanges sees a fossil add (rid=0, chnged=0) without a scan, since rid=0 now always means added (#231).

* fix(checkout): treat a non-file at a tracked path as missing (#232)

ScanChanges read each tracked path and failed the whole scan when a directory stood where a file belongs. Fossil's scan reports that path as NOT_A_FILE under its missing filter. The scan now stats first and treats anything that is not a regular file as missing, so Status reports it instead of erroring.

loadVFileRows orders rows by pathname, as fossil's changes lists them. The changelog notes that checkouts written before #231 may hold rid=0 placeholder rows, which HasChanges now counts. check-in: ff9c1af49d user: [email protected] tags: trunk

* fix(checkout): refuse to load a version with missing content (#231)

LoadVFile wrote a vfile row with rid=0 for each file whose content the repo does not hold (a phantom, or a delta on one), as a partial sync leaves it. In fossil's vfile rid=0 means "added", so those rows were indistinguishable from pending adds, and Extract then panicked in content.Expand(0) trying to write them.

LoadVFile now does what fossil's checkout does: it resolves every file first and refuses the version, naming up to ten missing files and wrapping errMissingContent, before changing vfile at all. It no longer returns a missing count, which no caller used.

Create is the one caller that tolerates the refusal. A tip that a partial sync has not finished is no reason to fail CreateCheckout: the caller may be about to extract an older, complete version (EdgeSync's ExtractTo). vfile then stays empty instead of claiming files the repo cannot produce. Create's environment defaulting, shared with Open, moves to resolveEnv, and the initial load to loadInitialVFile.

Tests cover the refusal (vfile untouched, files named), Extract leaving the checkout as it was, Create on a partial tip followed by Extract of a complete version, and parity with the fossil binary, which refuses the same repository with "missing content".

* fix(checkout): refuse missing content in Update before writing (#231)

Update wrote changed files to disk and only then reloaded vfile, which now refuses a version with missing content. A target sharing a missing file with the current version (its content is never read, because it does not change) could therefore leave disk on the target while vvar and vfile stayed on the old version. Like fossil's update, Update now resolves the target's content before writing anything, through resolveFiles, the resolution step split out of LoadVFile.

Update moves its target selection into updateTarget, reuses Extract's checkinTime, and builds its result through sortedUpdateResult, bringing it under 70 lines. Create now reports a tip it could not load to the observer instead of tolerating it silently.

* fix(checkout): report Update refusals to observers; tighten checks (#231)

The missing-content check in Update moves after ExtractStarted, so a refused update still reaches observers as an ExtractCompleted carrying the error, as other Update failures do. A negative target from the public UpdateOpts is an error, not a panic. resolveFiles errors carry their own prefix instead of LoadVFile's.

Adds the entry and exit assertions the new helpers were missing, a compile-time check on missingNamesMax, and wraps lines over 100 columns; tests check Version's error instead of discarding it. check-in: a317896683 user: [email protected] tags: trunk

* fix(blob): always inflate blob.content on load (#235)

blob.Load decided whether a row was stored raw by comparing len(content) with blob.size. That guess is unsound for delta rows: blob.size is the expanded artifact's length, while content is the compressed delta, and the two can be equal by coincidence. When they are, Load handed back the still-compressed bytes, and the delta applier failed with "expected integer at pos 0".

Go 1.27 replaced compress/flate's encoder, so the compressed size of any given delta shifted. The fixture in TestExpand_SingleDelta and TestIsAvailable_GroundedChain now compresses to exactly the target size (62 bytes), which exposed the bug. Any real repository could hit it under either Go version.

Fossil itself never makes this guess: content_of_blob (src/content.c:218-230) inflates blob.content unconditionally, and every writer here stores the compressed form. Load now does the same. TestLoadInflatesWhenStoredLengthEqualsSize forces the coincidence directly so it fails on the old code under any Go version.

Separately, TestDecodeRejectsOversizeDeclaredLength built its container from a 64-byte payload. Go 1.27's flate skips match-finding for inputs under 128 bytes, so that payload came out as a stored block larger than its input and tripped the helper's did-it-compress guard. The test now uses 1 KiB. The property it asserts is unchanged.

* docs(changelog): note the blob.Load inflate fix (#235) check-in: 60ef41b5b1 user: [email protected] tags: trunk

* fix(checkout): check Extract's safety before replacing vfile (#230)

Extract called LoadVFile(rid, clear=true) first and only then checked for local work. A refusal therefore returned with the current version's rows already gone: pending adds, renames and merge state were lost, and vvar still named the old version while vfile held only the new one, so Status saw an empty checkout. The check also compared disk against the target version, so switching a clean checkout to any version that changed a file was refused.

Extract now follows fossil's checkout command, before touching anything: the current version must have no unsaved changes, and no file the current version does not track may sit where the target would write different content (fossil prompts there; a library refuses). Force skips both, as before.

The safety scan runs through a new refreshChanged, the observer-free core of ScanChanges, so an Extract does not report a scan of its own. Extract is split into extractRows, checkinTime and extractRow to bring it under 70 lines; behavior of the write loop is unchanged.

* fix(checkout): let Extract proceed when no change would be lost (#230)

Create records the tip as checked out before any file is written, so Create followed by Extract of an older version, into a directory that already holds it, saw every file differing from the tip as an unsaved edit and refused. That is EdgeSync's ExtractTo pattern.

The safety check now refuses only changes the extract would lose: a plain edit whose content already equals what the target writes is not lost, so it no longer blocks. Adds, removals, renames, merge states and edits to files the target does not write still refuse. The edit check and the untracked-file check share wouldClobber, and the target manifest is read once and walked in manifest order.

* fix(checkout): never relax Extract's check while a merge is pending (#230)

A vmerge row is unsaved work on its own. With every merged file already matching the target, the previous commit let Extract switch versions, dropping the merge and leaving the stale vmerge row to give a later commit a bogus merge parent. The content-equality relaxation now applies only when no merge is pending. check-in: c9d97efb90 user: [email protected] tags: trunk

Status on a checkout created by `fossil open` failed with "converting NULL to string" on vfile.mhash. go-libfossil treated mhash as every file's baseline hash; fossil only fills it while a merge is pending (the hash of mrid when mrid != rid) and judges a file against the blob that vfile.mrid names. A NULL-tolerant scan alone would not have fixed it: with no baseline the comparison panics, and skipping it would hide real edits.

ScanChanges now compares each file with blob.uuid for vfile.mrid and moves chnged through fossil's own transitions (vfile_check_signature): a pending add (mrid=0) becomes 1, unchanged and clean-merge states (0, 2, 4) become 1 on a difference, a reverted edit returns to 0, and other merge states are kept. Comparing against the artifact's own uuid also picks the hash algorithm per file, independent of hash-policy.

The non-forced Extract guard read mhash with its error discarded, so on a fossil checkout it skipped the check and overwrote local edits. It now uses the same baseline, propagates read errors, and refuses while a merge is pending (mrid != rid or chnged > 1).

Status and HasChanges treat a rename as origname not in ('', pathname), since fossil stores origname == pathname on unrenamed rows, and HasChanges now sees `fossil mv`. A malformed row returns an error instead of being reported as ChangeNone.

LoadVFile and Manage stop writing mhash, and Manage writes islink, which fossil's vfile schema declares without a default.

Tests drive real fossil-made checkouts: clean status, edits and adds, a pending merge (status, preserved merge state, extract refusal), local edits under extract, fossil mv, and a go-libfossil Add read back by both tools. check-in: 0069100efe user: [email protected] tags: trunk

The file jumped from Unreleased straight to 0.7.0 — neither 0.8.0 nor 0.8.1 was ever written up, and the hash-policy fixes had no entry. check-in: 4f17779bd3 user: [email protected] tags: trunk, v0.9.0

SeedConfig never wrote hash-policy, so naming fell back to deriving from the repo's own artifacts — and an empty repo has none, settling on SHA1 permanently. Seed sha3, which is what fossil init writes.

Clone drops the row the same way it drops project-code: a clone adopts its source's policy, and an absent row derives naming from the artifacts that arrive.

Moves the policy enum to db so the seed and hash.NamingFor read one definition, and updates fixtures that hardcoded SHA1 names to ask the repo how it names artifacts. check-in: 1c02c944ec user: [email protected] tags: trunk

Store hardcoded SHA1, so committing into a sha3 repo mixed hash families and made every tracked file read as CHANGED. Read hash-policy once per store and name new artifacts with the algorithm it selects.

Content already stored under the other algorithm's name keeps that name unless the policy forbids it (sha3-only, shun-sha1), so unchanged files don't churn their F-cards.

Also routes checkout.Manage and uv.Write through the same lookup, replacing their width-sniffing guesses — uv's read project-code, which is 40 hex chars in every repo, so it could only ever pick SHA1. check-in: 847c195286 user: [email protected] tags: trunk

Three tests started a real fossil server and killed it with Process.Kill() without waiting for the exit. Kill signals but does not reap, so the server could still be writing its -wal/-shm files while t.TempDir()'s cleanup walked the directory. A file appearing mid-walk fails RemoveAll with "directory not empty", which is how TestCloneFromRealFossilUncompressedReply failed on CI while passing locally -- it needs a loaded machine to lose the race.

internal/sync/integration_test.go already did this correctly, Kill followed by Wait, with a comment naming the same failure. This applies that established pattern at the three sites that lacked it.

Separately, the unit-test step ran with -timeout=120s, a per-package budget. CI runs the package binaries concurrently, so under contention internal/manifest and internal/sync exceeded it and panicked with "test timed out" despite finishing in 19s and 34s on an unloaded runner. Raised to 300s, matching the DST step, so the margin does not depend on how busy the runner is. check-in: 2256f5d7ce user: [email protected] tags: trunk

go.work.sum is only consulted for workspace-mode builds. CI never uses the workspace -- every step in test.yml runs GOWORK=off -- so the file earns nothing in the repository and re-churns on any local dependency change. Go regenerates it on demand for anyone building with the workspace active, so untracking it costs nothing and removes the churn.

docs/superpowers was added to .gitignore, but four files under it were already tracked; an ignore rule does not untrack what is already in the index, so the directory kept showing up in diffs regardless. Untrack it so the rule actually takes effect.

Both files stay on disk. Also terminates .gitignore with a newline. check-in: fd413ecc56 user: [email protected] tags: trunk

Keep validated receive bytes in a bounded session linker so clone does not expand and hash every artifact again during finalization. Preserve bounded retry, cancellation, replay, and residual-repair behavior.

Closes #214 check-in: 7b6a67fb81 user: [email protected] tags: trunk

Route direct check-ins and internal rebuilds through canonical mlink derivation. Update sparse-history consumers and preserve legacy conflict-fork rows with an explicit RID discriminator migration.\n\nCloses #213 check-in: 04f41a4c9c user: [email protected] tags: trunk

Cloning the Fossil SCM corpus (67.6k blobs) took 293-302 s at 1251-1287 MB against canonical fossil's 20.8 s at 205 MB. A CPU and allocation profile of one clone found the cost was not delta expansion or hash verification.

27.2% of all CPU was a single un-memoized statement. content.AvailabilityCache memoizes the delta-chain walk but not the "SELECT rid FROM blob WHERE uuid=?" that starts it, so manifest.missingCheckinRefs re-seeked the blob.uuid index once per F-card of every check-in -- the same file hash resolved again for every revision that did not change it. That index is far larger than SQLite's page cache, so each seek was three page reads: pread syscalls were 34.7% of total samples. The walk it guards was under 1% of its own cost.

Separately, materializing 8.27 GB of content allocated 360 GB. delta.ApplyInto starts at a 64 KiB cap deliberately -- the target length is a wire claim and a few bytes can declare 4 GiB -- and then climbs to the real size through append's ~1.25x large-slice growth, ~3.4x the target in discarded intermediates, leaving slack that content.Cache.clip copied away again (203.9 GB and 59.8 GB respectively).

Three changes:

- Memoize uuid -> rid in AvailabilityCache. Hits only. Within one crosslink run a hit cannot go stale, because a sweep and a cascade only ever add blob rows; a miss can, because ridOrPhantom and blob.StorePhantom create rows mid-run, and remembering an absence would leave an artifact deferred after the content it waits on arrived. The scope that makes this sound is now stated on the type, and TestAvailabilityCacheMustNotSpanAPurge pins its one boundary: shun.Purge deletes blob rows, so a cache must not span one.

- Size delta.ApplyInto's output exactly. scanOutputSize walks the command stream first without allocating, checking every copy against bytes the source holds and every insert against bytes the delta carries, so the size it returns is a measurement rather than the header's claim. Any stream it cannot walk to a clean terminator falls back to the existing growth path, so malformed and hostile deltas allocate and fail exactly as before. The buffer now arrives tightly sized, which also makes clip's copy a no-op.

- Cut crosslinkCacheBytes from 256 to 128 MiB. Each byte of that budget costs roughly five bytes of peak RSS -- GOGC doubles it into the heap goal, and the scavenger is paced off GC cycles, of which a faster sweep runs fewer. At 256 MiB the first two changes gave 205.7 s but 1473 MB, faster at the cost of memory. The measured curve (32/64/128/256 MiB: 369.6 s/318 MB, 292.3 s/447 MB, 249.4 s/812 MB, 205.7 s/1473 MB) puts 128 MiB where both axes improve.

Result on the same corpus: 245.4 s at 780 MB, from 293-302 s at 1251-1287 MB -- 17% less wall time and 39% less peak RSS, with instructions retired down from 3.37T to 2.48T. The gap to canonical narrows from 14.1x/6.3x to 11.8x/3.8x.

Derived tables are unchanged against an unmodified build cloning the same server: blob 67530, event 27099 (identical per type), mlink 104919, plink 23741, tagxref 51766, leaf 1426, filename 1496. The corpus parity gate passes.

What remains is structural rather than a missing optimization: a clone expands every blob twice, once in the receive-time phantom-fill cascade and once in the final whole-repository sweep, where canonical crosslinks each artifact from content it already holds in hand at receive time. Cache.Expand splits 45.4 s / 35.3 s across those two passes. Collapsing them is a change to the receive path and belongs in its own issue. check-in: 8b16c91f5c user: [email protected] tags: trunk

A full clone of the Fossil self-hosting repository disagreed with what canonical fossil derives from the SAME artifacts: mlink -3,351 rows, tagxref -632, plink -5, leaf -1. `fossil rebuild` on our own clone put every one of them back, so the artifact set was complete and the fault was entirely in crosslink-time derivation.

Six independent causes, each verified against `fossil rebuild` on a 67.6k-blob corpus clone:

mlink, -3,351. An mlink row records a parent-to-child TRANSITION, and a merge check-in has more than one parent. Canonical runs the same file-level diff once per parent -- primary, then each merge parent, then each cherrypick source -- and flags the extra rows isaux=1; we only ever diffed against the primary parent. add_mlink is now ported whole (internal/manifest/addmlink.go), which also brings the two passes the old code approximated: the delta-vs-delta revert/resurrect pass, and the baseline-child deletion pass with its mperm=0 delete rows. "Added by merge" (pid=-1) follows canonical's rule too -- a count over the rows a file produced, not a name lookup in the merge parents. A parent whose manifest cannot be read now yields NO rows rather than a whole tree of bogus "added" rows, and the rows are derived later from the parent's side once it arrives (linkPlinkChildren), so the result no longer depends on the order the sweep visits artifacts in.

tagxref, -612 of -632. A check-in's T-card does not have to name the check-in carrying it: `fossil merge --integrate` records the branch it closed as `T +closed <hash>` on the merge check-in itself. Canonical applies a check-in's, a control artifact's and a technote's T-cards through one code path; we applied only the self-referential ones on check-ins and none at all on technotes.

tagxref, the rest. A wiki-

or event-<id> tag's value is the W-card body measured after LEADING whitespace is skipped, not the raw card length.

Artifact identification. The sweep parses every blob in the repository, so what counts as an artifact decides which rows exist at all. Three gaps: a bare card letter with no arguments is legal (fossil's own first check-in spells its parentless-ness as a bare "P" card) and rejecting it dropped that check-in entirely; an N-card after a P-card is legal in a technote for historical compatibility, and rejecting it dropped real technote revisions with their event-<id> and bgcolor tags; and the check-in-versus-control decision turns on a comment, files or parents, which made `merge --integrate` check-ins read as control artifacts. Fossil's required-card table now gates linking (isArtifact), on both the sweep and the dephantomize cascade, so a file whose first line parses as a Z-card no longer links as a dateless check-in and a wiki revision missing its U-card no longer links at all.

plink, -5. The ancestry a check-in declares is a property of the check-in, not of what happens to be in the repository when it is crosslinked. A parent hash with no blob yet now reserves a phantom the way canonical's uuid_to_rid does, so the edge exists whether the parent arrives in a later round or never; skipping it lost the edge for good, because nothing revisits an already-linked artifact.

leaf, -1. Nothing calls leaf_rebuild() outside `fossil leaves --recompute`; the table is maintained by leaf_check() over the rids leaf_eventually_check() schedules -- each rid a branch tag lands on, plus that rid's parents. Drawing candidates from event instead both missed phantom parents (which fossil does list) and added check-ins that carry no branch tag and no ancestry (which it does not).

After: a fresh corpus clone and `fossil rebuild` of that clone agree row for row on mlink, plink, tagxref, leaf AND event -- 104,919 / 23,741 / 51,766 / 1,426 / 27,099 either way, and `fossil test-integrity` reports 0 errors.

Not addressed here: the local commit path (manifest.insertMlinks) still writes one mlink row per file rather than per change, and does not record merge-parent transitions. That is a separate divergence from the clone parity this issue is about.

Co-authored-by: Dan Mestas <[email protected]>

A pull into an already-up-to-date clone of the Fossil SCM repository took 2m44s and moved nothing. A CPU profile of that no-op pull puts 134.4s of 149.1s of samples (90%) inside manifest.CrosslinkContext, the whole-repository sweep Sync runs after the round loop. Nothing in the protocol path registers.

Two independent costs, both O(repository size) and neither bounded by what arrived:

collectCrosslinkCandidates 99.5s (67%) linkBatch -> Cache.Expand 34.9s (23%) of which hash.SHA3 26.0s (17%)

The query was quadratic. Its four exclusions were correlated NOT EXISTS subqueries, and tagxref is indexed on (rid, tagid) and (tagid, mtime) -- neither covers srcid -- so SQLite rescanned all 51,134 tagxref rows once per blob, 67,615 times. EXPLAIN QUERY PLAN showed the CORRELATED SCALAR SUBQUERY -> SCAN tx. Rewritten as NOT IN, each set materializes once and is probed: identical 39,727 rows, 47.7s -> 0.08s measured under canonical fossil's SQLite.

The candidate set never shrank. A blob qualifies when it has none of the derived rows a crosslink writes, and ordinary file content never gains those rows however often it is examined -- so 39,727 of the repository's 67,615 blobs were re-expanded (delta chain walked, SHA3 re-verified) and re-parsed on every sync. A probe run confirms the sweep linked exactly 0 artifacts on each of two consecutive runs over the converged repository, at 3m20s and 3m15s.

Canonical fossil has no equivalent cost because it never sweeps: xfer.c attempts one crosslink per blob as it arrives, artifact or not. This gives the sweep the same once-per-blob property via crosslink_nonartifact, an on-demand table recording blobs that expanded but hold no parseable artifact. Both facts are properties of immutable blob content, so the verdict cannot go stale; a blob that failed to expand, or that parsed but was deferred, is deliberately not recorded and is reconsidered next sweep.

Measured on a clone of the Fossil SCM repository (67,615 blobs), no-op pull:

before 2m43.9s after 0.40s

Clone is unchanged at 5m30s (it pays the one-time examination either way), and the two clones' derived tables are identical -- blob, event, mlink, plink, tagxref, filename and leaf totals all match row for row.

Fixes #202 check-in: 3748b95de7 user: [email protected] tags: trunk

* fix: make authenticated sync interoperate with canonical fossil (#203)

Every credentialed sync against a real fossil server was answered "login failed", whether or not the password was right. Push always needs a login, so push was unusable against any real server, as was pull from anything that did not grant anonymous read.

The login algorithm was never the problem. Two wire-level defects were, and each on its own produces that identical, password-independent rejection.

The push and pull cards carried their two codes in the wrong order. The canonical form is `push SERVERCODE PROJECTCODE` — canonical's own client writes `blob_appendf(&send, "push %s %s\n", zSCode, zPCode)` and both ends read the project code out of the second argument (src/xfer.c). We emitted and parsed project code first. A clone therefore adopted the server's server-code as its project code, and since a login signature is salted with the project code, every signature was derived from the wrong value. The same inversion put the server code where a server validates the project code, so an authenticated push was refused with "wrong project" even once the login worked.

The nonce covered a random comment that was never transmitted. The nonce is the hash of every byte following the login card, and the server re-hashes those bytes to check it (src/xfer.c check_tail_hash) — so the comment supplying the randomness has to be on the wire, not merely in the hash. We appended it to the hashed payload only, producing a nonce the server could not reproduce; it rejected the login before comparing the password at all, which is why a wrong password looked exactly like a right one. Comment lines now have a card type, so they can be sent as well as hashed.

Two smaller faults on the same path: a clone treated the server's round-0 "not authorized to clone" as fatal, ending the session before credentials were ever offered, where canonical tolerates that error on the first cycle precisely so the client can retry with the login card it can only build once it knows the project code; and the sync path signed with SyncOpts.ProjectCode rather than the code it had already resolved from the repository, so a caller that let the repo supply its project code — as the push and pull cards allow — panicked instead of authenticating.

The password derivation itself, sha1("<project-code>/<login>/<password>"), was correct and is unchanged. It is now pinned two ways: against the hash the fossil binary actually stores in user.pw, and against a login card captured from fossil 2.28 on the wire. Canonical's rule that an already hashed 40-hex password is used as-is is now honoured too.

Regression coverage stands up a canonical fossil server with anonymous capabilities removed, so a login card is the only way in, and asserts that a credentialed clone transfers content, a wrong password is refused, and a local commit pushed under authentication lands in the server's blob table and on its timeline. Nothing in the suite authenticated against real fossil before; clone was only ever exercised against servers granting anonymous read, where the login card is never sent, which is how this shipped.

Four xfer codec expectations that pinned the inverted argument order are corrected to the canonical bytes.

* fix(sync): correct the nonce doc to match what the server recomputes

The comment described the nonce as covering the encoded non-login cards and the signature as using the stored password hash. Both readings are wrong in ways that matter to anyone changing this code: the nonce is defined over the bytes that actually follow the login card on the wire, because the server recomputes it from what it received (src/xfer.c check_tail_hash), and the signature is salted with the project code rather than derived from user.pw directly. Hashing bytes the message does not carry produces a nonce the peer cannot reproduce, which is exactly the password-independent rejection this fix was chasing. check-in: a745f4f2ac user: [email protected] tags: trunk

AddTag, ApplyTag and ApplyTagWithTx each carried their own copy of the same type guard and eight-argument propagate call. #198 had to apply its mtime guard to all three, and a fourth site -- or one site updated while the others are missed -- reproduces exactly the shape that produced #197 (leaf derived two different ways) and #199 (project code resolved two different ways).

Extract propagateAfterInsert so the guard travels with the propagation instead of being restated at each caller, and cover the consolidated path. check-in: 4806de7907 user: [email protected] tags: trunk

Branch tag propagation left 1,155 of 20,859 check-ins on a clone of the fossil repository carrying a different branch value than canonical, and tagxref at 68,883 rows against canonical's 51,764. Two independent defects in the port, both confirmed against fossil's own source and a canonical rebuild of the same blobs.

tag_insert is mtime-guarded and our tag application was not. Canonical opens it with

SELECT 1 FROM tagxref WHERE tagid=%d AND rid=%d AND mtime>=:mtime

and returns without touching tagxref -- and without calling tag_propagate -- when it matches (src/tag.c:173-186). AddTag, ApplyTag and ApplyTagWithTx all wrote unconditionally, so a check-in's own inline T-cards clobbered a later control artifact that had retagged the same check-in, then re-propagated the stale value to every descendant. On the real corpus one branch rename did this 19,115 times: check-in ab0a0d76 declares branch dual-license, three control artifacts later rename it to clear-title, and we reinstated dual-license and flooded the repository with it. The guard makes the outcome independent of the order the sweep visits the two artifacts in.

repairTagPropagation seeded from the wrong node. tag_propagate's per-child test is strict --

coalesce(srcid=0 AND tagxref.mtime<:mtime, 1) AS doit

-- so a descendant that already carries the tag holds it at exactly the origin's mtime, doit is 0, and the walk neither retags that child nor queues it. Replaying from tag origins therefore stopped dead at the first already-tagged descendant, stranding any check-in linked by a later sweep. Canonical seeds from the primary parent instead, right after inserting a new check-in's plink edges (src/manifest.c:2300-2302 and 2467-2469), which only ever tests that parent's own children. Seed from every primary parent, which is the sweep-order-independent equivalent. Worth 693 missing propagated branch rows.

Measured on a clone of the fossil repository against a canonical rebuild of the same blobs:

divergent branch values 1,155 -> 1 (of 20,858 compared) tagxref rows 68,883 -> 51,132 (canonical 51,764) leaf rows 1,463 -> 1,425 (canonical 1,426)

The residual 632-row tagxref gap is 613 missing `closed` singletons plus small change; it measures identically before and after this change and is a separate defect. The one remaining divergent branch value is a NULL-vs-empty-string difference in cancel-tag values, also pre-existing.

repairTagPropagation still runs before repairLeafTable, as #197 requires. check-in: 7043d6e3a9 user: [email protected] tags: trunk

test.yml had zero references to wasm or GOOS=js, so the WebAssembly build was never compiled on any leg. Browser support is the reason the ncruces driver is in the tree, and a break there would have shipped undetected -- including in db/config_wasm.go, whose js-tagged wasmPragmaOverrides() no job compiled at all.

Adds a build-only job covering 31 of the main module's 33 js-resolvable packages plus the ncruces driver submodule. The package set comes from go list rather than a hand-written list so new packages are gated by default. Two main packages are excluded, both for the same structural reason: cmd/libfossil and examples/embed-repo-api import the modernc driver directly in their own main.go, and modernc.org/libc has no js/wasm build. internal/stash and internal/undo declare //go:build !js in source and are dropped by go list itself. Building under -tags test_ncruces keeps internal/testdriver in the gate by pointing it at the driver that can actually target WASM.

Running tests under WASM needs a browser harness (wasmbrowsertest) and is deliberately out of scope; compilation catches the realistic failure mode. check-in: 10abca922c user: [email protected] tags: trunk

* fix(sync): derive the project code from the repo instead of panicking (#199)

Supplying User/Password made SyncOpts.ProjectCode mandatory in practice: buildLoginCard passed the caller's raw value straight to computeLogin, which asserts it is non-empty, so an authenticated Sync on a repo whose project code the caller had not repeated took the process down. Nothing in the public API signalled that a password made the field required, and a server embedding the library died with it.

buildRequest already fell back to the repo's stored project-code for the push/pull cards; that fallback was local to one function, which is exactly how the login path drifted out of step with it. Resolve the code once when the session is built — caller value first, the repo's own config second — and let buildRequest and buildLoginCard both read it. A code that genuinely cannot be determined now returns a descriptive error from the site that needs it rather than panicking.

Sync's deferred result-not-nil assertion fired during panic unwinding, where err and result are both nil, replacing the original cause with its own message. The invariant only holds on a normal return, so it no longer asserts while a panic is in flight.

* fix(sync): write received artifacts' visibility inside the storing transaction (#200)

Receiving an artifact cost three write transactions, not one. The blob was stored in a transaction, its dephantomize cascade committed another, and then applyPrivateStatus re-derived the rid on the pool and issued a bare DELETE FROM private -- an implicit transaction of its own, with its own BEGIN IMMEDIATE and fsync, for a row that a transaction microseconds earlier had guaranteed did not exist. Every igot card announcing an artifact already held cost one more. Measured against a real fossil peer: 1509 statements executed outside a transaction to receive 1504 artifacts.

That is minutes of pure commit overhead on a large repository -- roughly 200,000 write transactions for a 67k-artifact pull -- and it is fragile as well as slow. Each of those writes contests the database's single write lock independently, so N artifacts means N chances to lose that contest and fail outright rather than one chance to wait at a single BEGIN.

Visibility now travels with the content it describes: storeReceivedFile takes the visibility a private card announced and records it inside the transaction that stores the blob, using the rid that transaction already has. The status is now atomic with the content rather than a separate write that could be interrupted between the two. The igot path buffers its updates and writes them once per round. The same three-transaction shape on the server's receive path goes with it.

The same real-fossil pull now executes 2 statements outside a transaction for those 1504 artifacts, down from 1509. db.DB.PoolExecs makes that budget assertable, which is how the regression test pins O(rounds) rather than O(artifacts) without needing a large corpus.

On the failure originally reported: the "database is locked" error came from a harness that copied a live SQLite database out from under a running server without its WAL, so that specific lock holder is not a product defect and this does not claim to fix it. The transaction storm above is a real performance and robustness defect on its own evidence, and is what this changes.

* test(sync): pin the transaction budget, not just writes outside a transaction

The budget tests measured PoolExecs, which counts statements issued outside a transaction. That pins one half of the defect and misses the other: moving a per-artifact write from the pool into its own WithTx leaves PoolExecs unchanged while every artifact still pays a BEGIN IMMEDIATE and an fsync, and still contests the single write lock on its own. That is the cost issue #200 actually measured -- roughly three write transactions per received file, about 203,000 for one clone-sized pull.

Verified by mutation: reverting the batched visibility flush to one transaction per update left both budget tests green. They now count transactions too, via a new db.DB.WriteTxns counter, and the same mutation fails with "began 51 write transactions for 50 igot cards over 1 rounds; want <= 4".

Also stop the contention test from starving the pull it is meant to observe. The rival connection re-acquired the write lock immediately on release, so on a loaded machine it could hold the pull's single BEGIN past busy_timeout and fail the test even when the batching under test is perfect -- which is what happened on CI. It now yields between bursts, modelling another process doing occasional writes rather than testing SQLite's fairness. check-in: 18bfbc166c user: [email protected] tags: trunk

* fix: derive leaf from fossil's branch rule and bound the deferral rollup

Leaf derivation dropped 66% of rows on the real Fossil repository (487 of 1,426) because both leaf paths treated any parent link as retiring a leaf. Canonical fossil retires a check-in only when a child carries the *same* branch name, with an absent branch tag reading as "trunk" (src/leaf.c: is_a_leaf / leaf_check / leaf_rebuild all share that predicate and none consult plink.isprim). Verified against fossil 2.28 itself: after a branch commit off the trunk tip, both the incremental table and `fossil rebuild` keep the trunk tip as a leaf even though its only child is its primary child, and a branch tip merged into trunk stays a leaf as well.

Two code paths were wrong, in the same way:

- repairLeafTable excluded every `pid` in plink, so merged branch tips disappeared -- the class that accounts for 939 of the corpus's leaves. - Checkin deleted every parent from leaf, so committing onto a new branch silently retired the branch point.

Both now share leafCheck/repairLeafTable in internal/manifest/leaf.go. The recompute also has to run after tag propagation rather than before it: a check-in that declares no branch of its own only has a branch value once propagation has written it, and comparing against a missing value made every inherited-branch tip look like a leaf.

Applying the new predicate to the real repository's own event/plink/tagxref yields 1,428 against fossil's 1,426, and every one of the six differing rows is fossil's incremental bookkeeping rather than its predicate: 2 are phantom rids fossil's leaf_check inserts as plink parents of check-ins it linked, and 4 are parentless check-ins with no branch tag, which nothing ever enqueues for a leaf check. Running fossil's own leaf_rebuild() on that repository gives 1,494, so it is leaf_check, not leaf_rebuild, that the shipped table reflects.

Also caps the deferral rollup's identifier lists (issue #194). A clone deferring 180 check-ins on 417 missing blobs rendered a single ~28 KB log line; the counts stay exact at INFO and the lists print eight entries plus an "...and N more" so nothing is dropped without saying so.

Fixes #189 Fixes #194

* chore: retrigger code scanning after branch restore check-in: df495ba69e user: [email protected] tags: trunk

A clone of a large repository is a conversation of dozens of rounds spanning minutes. A single transient disconnect anywhere in it aborted the whole operation with "http transport: Post ...: EOF" and produced no store at all: there was no retry, no reconnect, and minutes of transfer were discarded for one dropped packet. Reported failures landed at rounds 6, 11 and 18 across otherwise identical runs, which is the signature of independent random drops rather than a deterministic defect.

Both round loops now send a failed round again after a bounded exponential backoff — four attempts, 250 ms doubling to a 2 s cap — but only when the failure was the network dropping the exchange. The classification is an allow-list, not a deny-list: a reply that arrived whole and would not decode, a card the peer rejected, a 4xx, or an unrecognised error from a Transport this package does not know all still fail on their first occurrence. Retrying those would turn a reproducible defect into an intermittent one, which is worse than the failure retrying exists to prevent.

Telling the two apart cannot be done reliably after the fact — a truncated zlib body and a truncated wire read both surface as io.ErrUnexpectedEOF — so the transports tag their own wire stages via the optional TransportFault interface, in the same shape as FramedTransport. Non-2xx replies are now reported as such instead of being handed to the decoder, which also stops a peer's outage from being misreported as a framing defect.

The caller's context governs throughout: a cancellation or deadline aborts immediately, including during a backoff wait, so retrying cannot extend an operation past the bound its caller set (#166, #167, #183).

Resuming a partially-completed clone across process exit is deliberately not included: it needs the clone cursor persisted, a staleness policy for the server's snapshot bound, and a separate entry point, and within-session retry already preserves the expensive part. check-in: b2928e9629 user: [email protected] tags: trunk

All three submodules required the root module at v0.6.4-0.20260724230440-48111c1c984d. That pseudo-version names a commit pushed on the #118 PR branch, which was squash-merged — rewriting the work as a new commit and orphaning 48111c1. `git branch -r --contains` finds it on no branch; it survives only in proxy.golang.org's cache.

CI never noticed because it resolves through the proxy. Direct-from-git resolution fails outright:

$ GOMODCACHE=$(mktemp -d) GOWORK=off go mod download github.com/danmestas/go-libfossil go: github.com/danmestas/[email protected]: invalid version: unknown revision 48111c1c984d

v0.7.0 is tagged and reachable, so the bootstrap constraint that forced a pseudo-version is gone. Repoint all three at the tag and tidy. The root go.mod hash is unchanged between the orphaned commit and v0.7.0, so this is a pure provenance fix with no dependency movement.

Add a release guard alongside the existing `replace` guard: any in-repo module required at a pseudo-version now fails the release before the tag ships. The pattern anchors on the 14-digit timestamp and 12-hex hash, and accepts either separator before the timestamp so the vX.Y.Z-0.<ts>-<hash> form is caught as well as v0.0.0-<ts>-<hash>. Third-party pseudo-versions such as ncruces/go-sqlite3-wasm are deliberately untouched. check-in: 9d35407ecb user: [email protected] tags: trunk

* fix: write the ticket event row with its tag (#184), quiet the cascade (#186)

crosslinkTicket applied the tkt-<uuid> tag and returned the event row as a pending item for a second pass that was a no-op stub, so the row was never written at all. collectCrosslinkCandidates excludes any blob that already has a tagxref row naming its own rid as srcid, so the tag the incomplete operation wrote also hid the artifact from every later sweep. Cloning fossil's own repository produced 96 event rows of type 't' against canonical fossil's 3,808 -- 100% of the D,J,K,U,Z ticket-change shape lost, with nothing logged and test-integrity still clean, because only derived tables were wrong.

Implement the ticket rebuild instead of reordering the writes: there was no event write to reorder. ticketRebuildEntry replays every artifact carrying the ticket's tag in mtime order and derives its ticket, ticketchng and event rows, mirroring fossil's ticket_rebuild_entry, ticket_insert and manifest_ticket_event. It runs inline in the transaction that writes the tag, so the marker meaning "already linked" and the rows that make that true commit together or not at all. Replaying the whole tag set per artifact also makes the result independent of arrival order.

The event wording, the ticket/ticketchng column mapping including '+field' append semantics, the %h escaping and the two hyperlink hash lengths were all derived from the fossil binary rather than assumed, and are pinned by TestFossilBinaryTicketParity: a corpus built with `fossil ticket add`/`change` is crosslinked by us, rebuilt by fossil, and the two derivations compared row for row. TestCrosslink_TicketArtifactsProduceEventRows is the count assertion whose absence let this ship.

Audited the other three exclusion predicates for the same split-write shape: forumpost and attachment are each written by the same function, in the same transaction, as the event row that accompanies them, and propagate errors, so neither can diverge. No change made to them.

Secondary (#186): AfterDephantomize warned once per blob that failed to parse as a manifest, which is the ordinary outcome for file content and what the whole-repository sweep has always skipped silently -- ~39k lines on a fossil clone, burying the failures that matter. It now skips them the way linkBatch does. A blob that parses but fails to link is still a real fault and still warns.

* docs: record the ticket rebuild's measured cost and why it stays inline

The inline rebuild is quadratic in one ticket's artifact count. The cheaper deferred shape stays atomic but needs two call sites to remember to invoke it, and a forgotten call reintroduces #184 silently. Write the measurements and that reasoning down at the function, so a later profile of a repository with one hot ticket does not turn into an optimization back into the bug. check-in: 39f646e0ae user: [email protected] tags: trunk