* fix(xfer): reject non-decimal card sizes and bound payload memory

A payload card's size was parsed with strconv.Atoi, so a peer sending `file HASH -1` reached readPayload's negative-size assertion and panicked the client or server decoding the message, and a signed size such as `+5` was accepted. Sizes now go through parseSize, which follows the sync draft's `decimal = 1*DIGIT` grammar and fossil's 32-bit limit and returns a decoding error for anything else. uvfile FLAGS use the same parser, so negative FLAGS are rejected as the draft requires. The assertion stays, now unreachable from peer input.

readPayload no longer allocates the declared size up front: its buffer grows with the bytes read, so a large size over a short message fails after allocating about what was sent.

A cfile payload may inflate only to its USIZE (a margin above it for a delta) and not past its 4-byte size prefix, and all cfile payloads in one message share a MaxCFileInflateBytes (1 GiB) budget, so a few megabytes of zlib cannot inflate to tens of gigabytes.

FuzzDecode exercises the decoder over both framings.

* fix(sync): refuse malformed xrow payloads and cap HTTP reply bodies

An xrow payload or xdelete PKData from a peer was decoded into a map and used as is. JSON null gave a nil map that reached coerceJSONNumbers' assertion, and a primary-key value of the wrong type or a nested value reached normalizeValue's, panicking the client or server. Both directions now decode through decodeRowJSON: one JSON object, nothing after it, and repo.ValidateRow's rules, which require scalar values and primary-key values that PKHash can normalize. A client refuses an xrow for a table it has not registered. Column types match case-insensitively, as table creation accepts them.

Both HTTP transports read replies through xfer.ReadBody, which stops at MaxBodyBytes (MaxDecompressedBytes plus framing headroom) and returns ErrBodyTooLarge, untagged for retry, instead of reading an endless body into memory.

FuzzTableSyncPayload drives the server handler and client over arbitrary payloads.

* fix(sync): check synced-table rows and identifiers before building SQL

repo.UpsertXRow wrote a row's map keys into its INSERT as column names, and the rows of an xrow payload or xdelete PKData come from the peer, so a key such as "val, mtime) VALUES(...) --" chose the statement's columns and values, and a key could select from other tables into a synced one. This held on the server for pushed rows and tombstones, and on the client for a server's rows.

A peer's row must now use exactly the table's declared columns: repo.ValidateRow refuses any other key, which also keeps a peer from setting _owner. UpsertXRow no longer trusts its caller: it reads the table's registered schema and refuses any key it does not declare, apart from _owner on an ownership-mode table. ListXRows, LookupXRowOwner and DeleteXRowByPKHash validate the table name and the column names they interpolate, so no synced-table identifier reaches SQL text unchecked.

* fix(sync): answer reqconfig only with fossil's transferable config

The sync server answered reqconfig NAME with any row of the config table, for any requester, including one with no capabilities, so project-code, server-code and every local setting could be read by anyone who could reach the sync endpoint.

handleReqConfig now follows fossil's page_xfer and configure.c. It answers only a /GROUP name, only for a requester with Read, matches the group by prefix in fossil's table order, and sends only the names fossil transfers in that group, in fossil's encoding. The user and subscriber tables also need Admin, which counts only for a verified login, and the concealed table needs RdAddr. A refused request gets no cards and no error.

The groups, their names and their capability rules live in one place, config_groups.go. A differential test answers the same requests with fossil's own server and with HandleSync on the same repository and requires identical config cards.

* test(sync): keep fossil http from leaving a backoffice process

* fix(sync): check the login nonce against the request body

The server verified a login card's signature over its nonce but never checked that the nonce was the SHA-1 of the request body after the login line, as fossil's check_tail_hash does (draft-fossil-sync-protocol-00 §6.2). A login card captured from one request therefore authenticated any other request body.

The decoder now records, for each login card, the SHA-1 of the decompressed bytes after the card's line, and VerifyLogin refuses a card whose nonce does not match it. A login card that was never decoded has no tail and is refused.

Login handling otherwise follows fossil's page_xfer and check_login: - a refused login ends the exchange with "error login failed" instead of continuing as nobody; - a second login card is refused with "multiple login cards"; - a successful login adds the user's capabilities to nobody's instead of replacing them; - nobody and anonymous log in without a password, and the reserved developer and reader logins and accounts with an empty password cannot be logged into.

Fixes GHSA-2g5p-q5f3-477p.

* fix(repo): seed nobody with fossil's default capabilities

A new repository seeded the nobody user with "cghijknorswy", so every request without a login card could push check-ins and unversioned files, write wiki pages and tickets, and held Setup. Seed the four built-in users `fossil new` creates, with fossil's capabilities: nobody "gjorz", anonymous "hz", developer "ei" and reader "kptw".

Setup and Admin now include check-in, read and clone, as they do in fossil, so the creating user can sync once given a password. A login to an account with no stored password is refused, as fossil's check_login refuses it; the creator starts with none, and the signature would otherwise be the SHA-1 of the nonce alone.

Tests that push without credentials grant nobody push explicitly. Existing repositories keep their stored capabilities.

* test(auth): reserved-login test updates the seeded developer and reader rows

New repositories now seed developer and reader as fossil does, so the test gives the existing rows a password instead of inserting them.

* docs(changelog): group the sync hardening fixes under Security

* test(sync): reqconfig tests sign their logins over the request body

The login nonce now has to cover the request body, so the reqconfig tests build their admin requests with signedRequest instead of a bare login card. check-in: 3a6126c3e9 user: [email protected] tags: trunk

The server answered a pull-only client's igot cards with gimme for artifacts it lacked. The client was not pushing, so it never sent them and kept going round after round until it hit the round cap. Fossil's page_xfer handles igot only when the client sent a push card, and does the same here: a pull-only igot just records what the client has.

The client also treated the server's gimme cards as outstanding work even when it was not pushing, so any server that did ask (an older libfossil, or another implementation) could hold a pull open. A session without push now ignores gimme cards.

Two handler tests relied on a pull card alone producing a gimme; they now send a push card. check-in: 1e3e2df2f2 user: [email protected] tags: trunk

* test(conformance): delta-format conformance suite (#262)

Check internal/delta against draft-fossil-delta-format-00, requirement by requirement: 39 requirements, 88 cases. Decoder cases (Apply and OutputSize) are byte vectors, each also given to fossil(1) through its SQL delta functions; fossil's verdict must match unless the case records why it follows the draft instead (fossil masks the high bit, does not verify the checksum, and returns the size as a signed int). Each rejection breaks one rule and keeps the trailer valid, so check order cannot hide the rule under test. Producer cases check Create against a strict oracle and against fossil's delta_apply, and fossil's delta_create against Apply.

FuzzDeltaApply compares Apply with an independent model decoder; FuzzDeltaRoundTrip checks Apply(Create(s, t)) == t. DF-3.1-6 (Create has no 2^32 bound) has no case: it needs a 4 GiB input.

* test(conformance): delta suite names the reason each rejection must give

A rejection case could pass because some other rule rejected the input. Every rejection case now names a substring of go-libfossil's error (or ErrChecksum), and the harness fails the case when the message differs.

Vectors that were rejected for the wrong reason are fixed or reattributed: copy-truncated (DF-4-2), reject-10-nul (non-empty count, so the NUL is the opcode), copy-overflow-64 (DF-4-11), and high-bit-count, which now tells a masking decoder from a conforming one (fossil accepts it). The phantom-base case uses a literal-only delta; the content-discard case checks ErrChecksum and the store-side header refusals move to DF-4.1-3. The allocation bound gets its own row (DF-6-5) and takes the least of five runs. DF-3.1-6 notes issue #279. Drops a duplicate size-query case, the sizeCase helper and the decorative fuzz guard bytes. check-in: 0629de2beb user: [email protected] tags: trunk

* test(conformance): artifact-format conformance suite (#263)

Adds the suite for draft-fossil-artifact-format-00 with errata 4.1.2, 4.2 and 4.6.4, so the parser, writer, naming, hash policy and delta manifest code are checked requirement by requirement instead of by scattered unit tests that each encode one reading of fossil.

- requirements/artifact.tsv: 101 requirements from Sections 3 to 8, pinned to the draft and the three errata. - Parse cases are a known-good base artifact per type plus one edit, with the Z card recomputed, so the edit alone explains a rejection. Every case also runs through fossil test-parse-manifest; fossil 2.28 agrees with all of them. Card legality, duplicates, arity and hash positions are generated from the bases and the draft's own tables. - Accepted inputs must re-encode byte for byte unless marked non-canonical. - API cases cover naming, verification, the comparator, the writer, the R card, the hash policy and delta-manifest enumeration. - FuzzArtifactRoundTrip checks parse/marshal/parse stability, and TestArtifactCorpus (CONFORMANCE_REPOS) parses every artifact fossil crosslinked in real repositories.

267 cases go-libfossil fails are marked with a "#0" placeholder in artifact_gaps_test.go, grouped by the code at fault, for issues to be filed against.

* test(conformance): tighten the artifact suite after review (#263)

A rejection only counts when the rule the case names makes it, so every rejection case that is not about the Z card must now pass deck.VerifyZ first. Inputs go-libfossil rejected only through the Z check (framing, residue, sterilizer, unstripped wrappers, an extra Z token) become gaps, along with VerifyZ itself, which rejects an absent Z card and reads the last byte where Section 6.2 says it must not.

- Cases can name the substring fossil's rejection reason must contain; the unknown-letter cases sit in sorted position, the sterilizer cases are filed under card order, which is what fossil rejects them for, and generated legality cases take their requirement from the draft's table. - The ratchet that never persists the hash policy is a gap: fossil opening the repository still reads auto. - New rows for 4.7.9, 6.2 (absent Z, last byte), 6.3.4 and 7.1. - The audit refuses the #0 placeholder, a test keeps the gap map and the fuzz test's forgiven losses tied to live gap cases, and one Z-card helper replaces two.

* test(conformance): mark artifact gaps with their issues (#263)

Replace the placeholder gap marks with the issues that track each fix (#265-#275), and keep coverage_test.go within 100 columns. check-in: d52a6fe050 user: [email protected] tags: trunk

Adds the conformance package that the per-draft suites build on. Nothing in the repository has yet checked go-libfossil against the Fossil drafts requirement by requirement.

- A Case names its requirement. A Gap (an issue) or Deviation (a reason) marks a known difference. A marked case still runs: while it fails it is skipped, and once it conforms it fails until the mark is removed, so the list of known failures cannot go stale. - Requirements tables pin the SHA-256 of the drafts and errata they were written against. Those are copied unmodified into conformance/rfcs by scripts/refresh-conformance-rfcs.sh. A changed draft fails the coverage test until its table has been re-read and its pin moved. - Coverage is derived from the cases rather than recorded: - COVERED: every case conforms. - PARTIAL: the table's row says what its cases leave out. - DEVIATION: a case is marked as a deliberate difference. - FAILING: a case is marked with an open gap. - GAP: no case yet.

CONFORMANCE_REPORT=<path> writes the requirement-by-requirement report as Markdown. - fossil(1) is run on every case's input, through test-parse-manifest, so a case's expectation is checked against fossil each time the suite runs. withZ appends the Z card, so a case that edits one card never also breaks the checksum.

The harness's own rules are tested in harness_self_test.go. No suites are registered yet; the artifact-format suite is next (#263). check-in: 57a558fcbf user: [email protected] tags: trunk

* fix(manifest): store check-in, link and tag rows as fossil does (#258)

A rebuilt or locally written repository differed from fossil's in three derived values: event and plink times in their last digits, event.omtime left NULL, and a valueless tag stored as '' instead of NULL.

Fossil stores a tag time as SQLite's julianday() bound as a double, but writes event, plink, attachment and forumpost times into SQL text with its own printf("%.17g"), which emits 16 digits in long double arithmetic. TimeToJulian is now julianday() exactly (whole milliseconds over 86400000) and TimeToTextJulian ports fossil's printf conversion, so both match fossil bit for bit. The conversion follows builds where long double is a double (macOS, Windows); fossil itself differs across platforms in the last digit.

Check-in events record omtime, tag writes bind NULL for an empty value, the readers of event and plink times under the ncruces driver rebuild the text form, and the CLI's tag listing reads NULL values.

* test(db): sweep both stored julian forms; scan the text form from driver times (#258)

* test(verify): compare text times bit for bit only where fossil computes them in double (#258) check-in: 2c2e5e74b6 user: [email protected] tags: trunk

* fix(sync): clone and sync exchanges follow the sync draft (#255)

* docs(changelog): clone and sync exchanges follow the sync draft (#255)

* fix(sync): fail unfinished clones; ask for held phantoms once (#255 review) check-in: e46a6c3f60 user: [email protected] tags: trunk

Annotate aligned each version with its parent using its own LCS, so where a line repeats (blank lines, braces) it could pick a different copy as the new one than fossil does, crediting the line to another check-in. It also trimmed every trailing newline, dropping trailing blank lines.

Annotate now uses a port of fossil's line diff (break_into_lines, diff_all, longestCommonSequence and the rest of diff.c that annotation uses), hash included, since the hash orders the candidates and so decides ties. Like fossil's annotate, it diffs each older version against the starting one and credits a line to the version just newer than the first ancestor that lacks it. This also drops the table the size of both versions that the old LCS allocated at every step. check-in: ada7aadbb5 user: [email protected] tags: trunk

Past the first parent, walkParentChain wrote each line's attribution by its index in an older version, so lines were credited to the wrong check-in and an older version longer than the start panicked. Carry each compared line's start-version index through the walk and stop following lines that no longer match. check-in: a63adf1dcf user: [email protected] tags: trunk

The CLI's merge marked a conflict with vfile.chnged=5, which fossil reads as "added by integrate", and recorded no merge-in version or vmerge row, so a merge went unseen and the next commit dropped the merge parent.

Merging now lives in the library as Checkout.Merge and records what fossil records: chnged=2 and mrid for a file taken from the merged-in version, chnged=3 for an added file, a per-file vmerge row for a merged file, and vmerge id 0 for the merge. Conflicts use fossil's markers and are found by looking for them (Checkout.Conflicts), so fossil and go-libfossil read each other's merges. Where fossil relies on its undo copy, Merge keeps the file or refuses instead.

Commit reads vmerge for merge parents, and, as fossil's commit does, refuses unresolved conflict markers, empty commits, unknown names and partial commits of a merge. The CLI's merge, conflicts, mark-resolved and ci commands call the library; ci now commits the checkout. check-in: 7ea6fba63f user: [email protected] tags: trunk

The release workflow's last step sent EdgeSync an upstream-bump event with a personal access token. It has failed with 401 on every release since v0.8.1 because the token expired, marking each release run failed after the release itself was published. EdgeSync also stopped accepting the event when the repository was renamed.

EdgeSync now tracks go-libfossil with Dependabot (danmestas/EdgeSync #199), which needs no token here, so the step is removed. A release run now succeeds once the GitHub release exists. check-in: af8e531e68 user: [email protected] tags: trunk

* fix(checkout): revert undoes a pending rename (#242)

Revert-all selected rows by content and removal, so a rename recorded by fossil mv (origname set, chnged 0) was skipped; a renamed and edited file had its committed content written under the new name, leaving the rename pending. A named revert only matched the current name.

Like fossil's revert, a pending rename is now undone: the file under the new name is removed, the row takes its old name back (UPDATE OR REPLACE, as fossil does), and the committed content is written there. Revert-all includes renamed rows, and a named revert answers to either the new or the old name. A renamed file a merge added is still removed outright, as before and as in fossil.

removeRegularFile is shared by the rename undo and removeMergeAdded, so neither deletes a directory the user made at that path.

* fix(checkout): plan a revert before changing anything (#242)

Undoing a rename with UPDATE OR REPLACE let a swap destroy files: reverting a.txt and b.txt exchanged through fossil mv deleted the other row, the loop then removed a file it had just restored from its stale row list, and the next restore failed half way, leaving both files gone from disk. A new file added under the old name was overwritten the same way. A named revert of the old name also stopped at the first match.

Revert now plans, checks, then acts. planRevert decides every target row's action (restore, un-manage, remove a merge-added file, undo a rename) before anything changes; checkRenameUndoFree refuses the whole revert when undoing a rename would land on a name another tracked row holds, since go-libfossil has no undo copy to recover an overwritten file (fossil saves one); revertFile carries out the planned action. The rename undo is a plain UPDATE, which fails rather than replace a row. A name selects every row it matches, new name or old. The callback reports a rename under the name the file gets back.

Swaps and chains, which fossil completes via its undo store, are #248.

* fix(checkout): revert named paths as one; refuse untracked file at old name (#242)

Review of the planned revert found two remaining ways to lose work:

- Several named paths were reverted one at a time, so in `revert b.txt z.txt` the edit to b.txt was discarded before z.txt's rename undo was refused. revertPaths now gathers every named row (new or old name) and plans and checks them as one revert: a refusal reverts none of them. - Undoing a rename restored the committed content over an untracked file at the old name. checkRenameUndoFree now also refuses when a file on disk there differs from the committed content (wouldClobber), for the same reason it refuses a tracked one: there is no undo copy.

* docs(changelog): note revert's untracked-file and multi-path rules (#242) check-in: 50d1a7f72b user: [email protected] tags: trunk

* fix(content): report database faults from the availability check (#239)

IsAvailable and AvailableByUUID return a bool, so a busy, closed or I/O-failed database read as "content missing". Where missing content is tolerated on purpose, that swallowed the fault: since #231, CreateCheckout leaves vfile empty for a partly synced tip, and a fault during resolveFiles' availability check took the same path, so Create succeeded with an empty checkout.

The chain walk moves to checkAvailable, which returns an error for a fault and a plain false for an unknown rid, a phantom, a cycle or an over-long chain. New CheckAvailableByUUID exposes that distinction, and resolveFiles uses it, so a fault fails LoadVFile with its own error instead of errMissingContent. IsAvailable and AvailableByUUID keep their signatures and their fail-closed reading of a fault; their ~30 callers treat "not available" as a failure already.

* docs(changelog): note the availability fault fix (#239)

* fix(content): NULL size is missing content; don't cache faults (#239)

checkAvailable scanned blob.size into an int64, so a NULL size (legal in the schema) became a hard error in resolveFiles where it used to read as missing content. Both walks now read a NULL size as a phantom.

AvailabilityCache memoized a walk's verdict for every node it visited, including a false that came from a database fault, so one transient fault stayed wrong for the rest of the crosslink sweep. A faulted walk now returns false without being remembered; an unknown rid is still a verdict and is cached as before. check-in: 4b14509374 user: [email protected] tags: trunk

* fix(checkout): dry runs change nothing; switching ends a merge (#236)

A dry-run Extract still called LoadVFile(rid, clear=true) and finalizeExtract, replacing the current version's vfile rows (pending adds, renames, merge state) and moving vvar to the target. A dry-run Update still ran finalizeUpdate, with the same effect.

Extract now lists the target's files from its manifest (resolveFiles) instead of reading them back from vfile, so a dry run never loads them; only a real extract replaces vfile and records the version. Update skips finalizeUpdate on a dry run.

finalizeExtract also clears vmerge, as fossil's checkout does: a forced switch used to leave a merge record that would give the next commit a bogus merge parent.

* fix(checkout): forced extract starts fresh; re-extract keeps a merge (#236)

Two gaps against fossil's checkout found in review:

- A forced extract dropped only other versions' vfile rows (LoadVFile keeps the target's), so re-forcing the current version left a pending rename's two names on disk and in vfile, and a removed file restored but still marked deleted. fossil checkout --force deletes every row first; replaceFileList now does the same when Force is set. - Extract cleared vmerge on every completed run. fossil's checkout returns early when the version does not change and Force is off, keeping a pending merge; finalizeExtract now ends the merge only for a switch or a forced extract.

* refactor(checkout): finalizeExtract reads the prior version itself (#236)

Only the final step needs the version being replaced, and nothing changes vvar before it, so finalizeExtract reads it there. A dry run no longer calls Version at all, and Extract drops to 64 lines. check-in: 139d426384 user: [email protected] tags: trunk

* fix(cli): add, rm, rename and revert call the library (#234)

These commands wrote the checkout's vfile with raw SQL, each with its own copy of rules the library already gets right, and ignored most Exec errors. rename recorded the new name but left the file under the old one, so the next commit failed on a "missing" tracked file. revert only cleared change flags over an edited file and restored nothing. add wrote a SHA1 mhash whatever the repo's hash policy.

They now open the checkout through the library (openWorkingCheckout, which finds the repository from the checkout when -R is not given) and call Add, Remove, Rename and Revert, returning every error, including from closing. rename moves the file, like fossil mv --hard; revert restores committed content, like fossil revert.

Two library gaps the CLI exposed are fixed where they live, matching fossil's revert: - Revert refreshes change flags from disk before choosing files, so an edit no scan has recorded yet is reverted instead of skipped. - Reverting a never-committed add un-manages it and leaves the file on disk. It used to delete the file, destroying the only copy; fossil prints UNMANAGE and keeps it.

revertFile's restore path moves to restoreCommitted, bringing it under 70 lines. The merge, merge-resolve and conflicts commands are #244.

* docs(changelog): note the CLI and Revert changes (#234)

* fix(checkout,cli): revert and rename match fossil; find repo without -R (#234)

Review of the CLI switch found the library and the new helper falling short of fossil:

- Without -R, every command failed on a fresh fossil checkout: the repository lookup opened the checkout database read-only, which clashes with the WAL pragma on a database fossil left in delete journal mode, and hid the real error. It now opens it as the library does, reports the error, and resolves a relative repository path, as fossil open stores it, against the checkout's directory. - Rename moved the file onto an existing untracked file, destroying it. It now refuses, as fossil mv --hard does, before changing anything. - Revert skipped a file missing from disk (no change flag records it) while the CLI reported it reverted. A named file is now reverted whatever its flags say, and revert-all includes missing files. - Revert left merge state behind: restored rows kept mrid on the merged-in version and vmerge kept its row, so the next scan saw the file changed and the next commit recorded a merge parent. Restored rows now reset mrid and mhash, and revert-all clears vmerge, as fossil's revert does.

Rename's vfile lookups move to renameRowID and revert-all's selection to needsRevert, keeping both functions under 70 lines. The re-add test now checks the vfile row itself, since fossil's own scan would hide a stray chnged=1.

* fix(checkout): revert removes a file a merge added, as fossil does (#234)

Revert decided between restore and un-manage by rid alone. A file a merge added has a non-zero rid (its merged-in artifact) but is not in the checked-out version, so revert restored it as if committed and left it tracked; the next commit then included it. Fossil looks the file up in the version's manifest and, when it is absent and rid is not 0, deletes the file and its row (its content lives on in the merged-in version).

revertFile now takes the vfile row and the version's path set and does the same: a pending add is un-managed and kept on disk; a file the version lacks, under its name or its name before a pending rename, is removed; anything else gets its committed content back. Checking the pre-rename name keeps a renamed, edited file from being deleted.

Rename's refusal message now fits a directory at the destination too.

* fix(checkout): never remove a non-file when reverting a merge add (#234)

removeMergeAdded deletes only a regular file at the merge-added path; anything else there, such as a directory the user made, is left alone. Revert's doc now says edits to reverted files are discarded: fossil saves an undo copy first, go-libfossil has no undo yet. check-in: de8f3ed302 user: [email protected] tags: trunk

* refactor(vfile): give the checkout's vfile encoding one owner (#233)

Checkin, stash and status each decoded fossil's vfile columns on their own: rid=0 for an add, origname==pathname on unrenamed rows, chnged merge codes. That is how #228 happened, and stash still had it wrong: it selected chnged=1 only, so files changed by a merge (chnged 2..9) were silently left out, and it stashed a rename as a plain edit.

New package internal/vfile owns the encoding. Load reads a version's rows; Row answers what callers ask (IsAdded, IsRemoved, RenamedFrom, ContentChanged, PendingMerge); Classify gives the change type in fossil's order. checkout.FileChange is now an alias of vfile.Change, so the checkout API is unchanged.

internal/checkout's status, Extract safety check and checkin read rows through it. The never-read vfileCommitEntry.changed field is gone.

internal/stash selects changes with vfile.Classify against the checked- out version and refuses a pending merge or rename by name, since its save and revert paths restore files by current name against the committed version and would drop either. Its store-and-revert loop is split into stashRecord and revertStashedFile; removed and modified files share one revert path, keyed by row id.

* fix(vfile): a pending merge is chnged 2-5 or mrid != rid (#233)

PendingMerge counted every chnged above 1 as a merge, but 6 through 9 record a change to the executable or symlink bit; fossil leaves chnged=6 after a plain chmod +x. Stash then refused an ordinary permission change as "a pending merge". It also ignored mrid: an edit made on top of a merge drops chnged back to 1 while mrid still names the merged-in content, so the refusal could be bypassed and the merge half-dropped.

Row now carries mrid; PendingMerge is chnged 2..5 or mrid != rid, and ModeChanged covers 6..9. Stash refuses a permission change with that reason, since its revert writes default permissions.

Load reads a NULL rid as 0 and a NULL mrid as rid, the tolerance the checkin code had when it scanned rid into a NullInt64.

* refactor(checkout): ask vfile.Row about merges in Extract's check (#233)

Extract's safety check relaxed only rows with chnged == 1, decoding the column itself. It now asks the row (not a pending merge, not a mode change), keeping fossil's encoding in internal/vfile. Equivalent for every real row. nextChangedState's comment no longer calls the mode changes (6..9) merge states. check-in: f55d0ed2de user: [email protected] tags: trunk

* fix(checkout): classify changes in one place; report missing files (#232)

HasChanges, VisitChanges and Extract's safety check each decided what counts as a change: two SQL WHERE clauses and classifyChange, with an error path whose only job was catching their disagreement. They had already drifted once (HasChanges missed renames until #229).

Now one reader, loadVFileRows, returns every row of a version and one classifier, classifyChange, judges it, in the order fossil's changes command uses: removed > missing > added > renamed > modified. A renamed file that is also edited stays "renamed", so callers that see only the change type do not lose the rename. The WHERE clauses and the malformed-row error are gone.

VisitChanges with a scan now reports a tracked file that is gone from disk as ChangeMissing, which the public API already exposed as "missing" but nothing produced; such files were left out, or shown as added or modified. HasChanges sees a fossil add (rid=0, chnged=0) without a scan, since rid=0 now always means added (#231).

* fix(checkout): treat a non-file at a tracked path as missing (#232)

ScanChanges read each tracked path and failed the whole scan when a directory stood where a file belongs. Fossil's scan reports that path as NOT_A_FILE under its missing filter. The scan now stats first and treats anything that is not a regular file as missing, so Status reports it instead of erroring.

loadVFileRows orders rows by pathname, as fossil's changes lists them. The changelog notes that checkouts written before #231 may hold rid=0 placeholder rows, which HasChanges now counts. check-in: ff9c1af49d user: [email protected] tags: trunk

* fix(checkout): refuse to load a version with missing content (#231)

LoadVFile wrote a vfile row with rid=0 for each file whose content the repo does not hold (a phantom, or a delta on one), as a partial sync leaves it. In fossil's vfile rid=0 means "added", so those rows were indistinguishable from pending adds, and Extract then panicked in content.Expand(0) trying to write them.

LoadVFile now does what fossil's checkout does: it resolves every file first and refuses the version, naming up to ten missing files and wrapping errMissingContent, before changing vfile at all. It no longer returns a missing count, which no caller used.

Create is the one caller that tolerates the refusal. A tip that a partial sync has not finished is no reason to fail CreateCheckout: the caller may be about to extract an older, complete version (EdgeSync's ExtractTo). vfile then stays empty instead of claiming files the repo cannot produce. Create's environment defaulting, shared with Open, moves to resolveEnv, and the initial load to loadInitialVFile.

Tests cover the refusal (vfile untouched, files named), Extract leaving the checkout as it was, Create on a partial tip followed by Extract of a complete version, and parity with the fossil binary, which refuses the same repository with "missing content".

* fix(checkout): refuse missing content in Update before writing (#231)

Update wrote changed files to disk and only then reloaded vfile, which now refuses a version with missing content. A target sharing a missing file with the current version (its content is never read, because it does not change) could therefore leave disk on the target while vvar and vfile stayed on the old version. Like fossil's update, Update now resolves the target's content before writing anything, through resolveFiles, the resolution step split out of LoadVFile.

Update moves its target selection into updateTarget, reuses Extract's checkinTime, and builds its result through sortedUpdateResult, bringing it under 70 lines. Create now reports a tip it could not load to the observer instead of tolerating it silently.

* fix(checkout): report Update refusals to observers; tighten checks (#231)

The missing-content check in Update moves after ExtractStarted, so a refused update still reaches observers as an ExtractCompleted carrying the error, as other Update failures do. A negative target from the public UpdateOpts is an error, not a panic. resolveFiles errors carry their own prefix instead of LoadVFile's.

Adds the entry and exit assertions the new helpers were missing, a compile-time check on missingNamesMax, and wraps lines over 100 columns; tests check Version's error instead of discarding it. check-in: a317896683 user: [email protected] tags: trunk

* fix(blob): always inflate blob.content on load (#235)

blob.Load decided whether a row was stored raw by comparing len(content) with blob.size. That guess is unsound for delta rows: blob.size is the expanded artifact's length, while content is the compressed delta, and the two can be equal by coincidence. When they are, Load handed back the still-compressed bytes, and the delta applier failed with "expected integer at pos 0".

Go 1.27 replaced compress/flate's encoder, so the compressed size of any given delta shifted. The fixture in TestExpand_SingleDelta and TestIsAvailable_GroundedChain now compresses to exactly the target size (62 bytes), which exposed the bug. Any real repository could hit it under either Go version.

Fossil itself never makes this guess: content_of_blob (src/content.c:218-230) inflates blob.content unconditionally, and every writer here stores the compressed form. Load now does the same. TestLoadInflatesWhenStoredLengthEqualsSize forces the coincidence directly so it fails on the old code under any Go version.

Separately, TestDecodeRejectsOversizeDeclaredLength built its container from a 64-byte payload. Go 1.27's flate skips match-finding for inputs under 128 bytes, so that payload came out as a stored block larger than its input and tripped the helper's did-it-compress guard. The test now uses 1 KiB. The property it asserts is unchanged.

* docs(changelog): note the blob.Load inflate fix (#235) check-in: 60ef41b5b1 user: [email protected] tags: trunk

* fix(checkout): check Extract's safety before replacing vfile (#230)

Extract called LoadVFile(rid, clear=true) first and only then checked for local work. A refusal therefore returned with the current version's rows already gone: pending adds, renames and merge state were lost, and vvar still named the old version while vfile held only the new one, so Status saw an empty checkout. The check also compared disk against the target version, so switching a clean checkout to any version that changed a file was refused.

Extract now follows fossil's checkout command, before touching anything: the current version must have no unsaved changes, and no file the current version does not track may sit where the target would write different content (fossil prompts there; a library refuses). Force skips both, as before.

The safety scan runs through a new refreshChanged, the observer-free core of ScanChanges, so an Extract does not report a scan of its own. Extract is split into extractRows, checkinTime and extractRow to bring it under 70 lines; behavior of the write loop is unchanged.

* fix(checkout): let Extract proceed when no change would be lost (#230)

Create records the tip as checked out before any file is written, so Create followed by Extract of an older version, into a directory that already holds it, saw every file differing from the tip as an unsaved edit and refused. That is EdgeSync's ExtractTo pattern.

The safety check now refuses only changes the extract would lose: a plain edit whose content already equals what the target writes is not lost, so it no longer blocks. Adds, removals, renames, merge states and edits to files the target does not write still refuse. The edit check and the untracked-file check share wouldClobber, and the target manifest is read once and walked in manifest order.

* fix(checkout): never relax Extract's check while a merge is pending (#230)

A vmerge row is unsaved work on its own. With every merged file already matching the target, the previous commit let Extract switch versions, dropping the merge and leaving the stale vmerge row to give a later commit a bogus merge parent. The content-equality relaxation now applies only when no merge is pending. check-in: c9d97efb90 user: [email protected] tags: trunk

Status on a checkout created by `fossil open` failed with "converting NULL to string" on vfile.mhash. go-libfossil treated mhash as every file's baseline hash; fossil only fills it while a merge is pending (the hash of mrid when mrid != rid) and judges a file against the blob that vfile.mrid names. A NULL-tolerant scan alone would not have fixed it: with no baseline the comparison panics, and skipping it would hide real edits.

ScanChanges now compares each file with blob.uuid for vfile.mrid and moves chnged through fossil's own transitions (vfile_check_signature): a pending add (mrid=0) becomes 1, unchanged and clean-merge states (0, 2, 4) become 1 on a difference, a reverted edit returns to 0, and other merge states are kept. Comparing against the artifact's own uuid also picks the hash algorithm per file, independent of hash-policy.

The non-forced Extract guard read mhash with its error discarded, so on a fossil checkout it skipped the check and overwrote local edits. It now uses the same baseline, propagates read errors, and refuses while a merge is pending (mrid != rid or chnged > 1).

Status and HasChanges treat a rename as origname not in ('', pathname), since fossil stores origname == pathname on unrenamed rows, and HasChanges now sees `fossil mv`. A malformed row returns an error instead of being reported as ChangeNone.

LoadVFile and Manage stop writing mhash, and Manage writes islink, which fossil's vfile schema declares without a default.

Tests drive real fossil-made checkouts: clean status, edits and adds, a pending merge (status, preserved merge state, extract refusal), local edits under extract, fossil mv, and a go-libfossil Add read back by both tools. check-in: 0069100efe user: [email protected] tags: trunk

The file jumped from Unreleased straight to 0.7.0 — neither 0.8.0 nor 0.8.1 was ever written up, and the hash-policy fixes had no entry. check-in: 4f17779bd3 user: [email protected] tags: trunk, v0.9.0

SeedConfig never wrote hash-policy, so naming fell back to deriving from the repo's own artifacts — and an empty repo has none, settling on SHA1 permanently. Seed sha3, which is what fossil init writes.

Clone drops the row the same way it drops project-code: a clone adopts its source's policy, and an absent row derives naming from the artifacts that arrive.

Moves the policy enum to db so the seed and hash.NamingFor read one definition, and updates fixtures that hardcoded SHA1 names to ask the repo how it names artifacts. check-in: 1c02c944ec user: [email protected] tags: trunk

Store hardcoded SHA1, so committing into a sha3 repo mixed hash families and made every tracked file read as CHANGED. Read hash-policy once per store and name new artifacts with the algorithm it selects.

Content already stored under the other algorithm's name keeps that name unless the policy forbids it (sha3-only, shun-sha1), so unchanged files don't churn their F-cards.

Also routes checkout.Manage and uv.Write through the same lookup, replacing their width-sniffing guesses — uv's read project-code, which is 40 hex chars in every repo, so it could only ever pick SHA1. check-in: 847c195286 user: [email protected] tags: trunk

Three tests started a real fossil server and killed it with Process.Kill() without waiting for the exit. Kill signals but does not reap, so the server could still be writing its -wal/-shm files while t.TempDir()'s cleanup walked the directory. A file appearing mid-walk fails RemoveAll with "directory not empty", which is how TestCloneFromRealFossilUncompressedReply failed on CI while passing locally -- it needs a loaded machine to lose the race.

internal/sync/integration_test.go already did this correctly, Kill followed by Wait, with a comment naming the same failure. This applies that established pattern at the three sites that lacked it.

Separately, the unit-test step ran with -timeout=120s, a per-package budget. CI runs the package binaries concurrently, so under contention internal/manifest and internal/sync exceeded it and panicked with "test timed out" despite finishing in 19s and 34s on an unloaded runner. Raised to 300s, matching the DST step, so the margin does not depend on how busy the runner is. check-in: 2256f5d7ce user: [email protected] tags: trunk

go.work.sum is only consulted for workspace-mode builds. CI never uses the workspace -- every step in test.yml runs GOWORK=off -- so the file earns nothing in the repository and re-churns on any local dependency change. Go regenerates it on demand for anyone building with the workspace active, so untracking it costs nothing and removes the churn.

docs/superpowers was added to .gitignore, but four files under it were already tracked; an ignore rule does not untrack what is already in the index, so the directory kept showing up in diffs regardless. Untrack it so the rule actually takes effect.

Both files stay on disk. Also terminates .gitignore with a newline. check-in: fd413ecc56 user: [email protected] tags: trunk

Keep validated receive bytes in a bounded session linker so clone does not expand and hash every artifact again during finalization. Preserve bounded retry, cancellation, replay, and residual-repair behavior.

Closes #214 check-in: 7b6a67fb81 user: [email protected] tags: trunk

Route direct check-ins and internal rebuilds through canonical mlink derivation. Update sparse-history consumers and preserve legacy conflict-fork rows with an explicit RID discriminator migration.\n\nCloses #213 check-in: 04f41a4c9c user: [email protected] tags: trunk

Cloning the Fossil SCM corpus (67.6k blobs) took 293-302 s at 1251-1287 MB against canonical fossil's 20.8 s at 205 MB. A CPU and allocation profile of one clone found the cost was not delta expansion or hash verification.

27.2% of all CPU was a single un-memoized statement. content.AvailabilityCache memoizes the delta-chain walk but not the "SELECT rid FROM blob WHERE uuid=?" that starts it, so manifest.missingCheckinRefs re-seeked the blob.uuid index once per F-card of every check-in -- the same file hash resolved again for every revision that did not change it. That index is far larger than SQLite's page cache, so each seek was three page reads: pread syscalls were 34.7% of total samples. The walk it guards was under 1% of its own cost.

Separately, materializing 8.27 GB of content allocated 360 GB. delta.ApplyInto starts at a 64 KiB cap deliberately -- the target length is a wire claim and a few bytes can declare 4 GiB -- and then climbs to the real size through append's ~1.25x large-slice growth, ~3.4x the target in discarded intermediates, leaving slack that content.Cache.clip copied away again (203.9 GB and 59.8 GB respectively).

Three changes:

- Memoize uuid -> rid in AvailabilityCache. Hits only. Within one crosslink run a hit cannot go stale, because a sweep and a cascade only ever add blob rows; a miss can, because ridOrPhantom and blob.StorePhantom create rows mid-run, and remembering an absence would leave an artifact deferred after the content it waits on arrived. The scope that makes this sound is now stated on the type, and TestAvailabilityCacheMustNotSpanAPurge pins its one boundary: shun.Purge deletes blob rows, so a cache must not span one.

- Size delta.ApplyInto's output exactly. scanOutputSize walks the command stream first without allocating, checking every copy against bytes the source holds and every insert against bytes the delta carries, so the size it returns is a measurement rather than the header's claim. Any stream it cannot walk to a clean terminator falls back to the existing growth path, so malformed and hostile deltas allocate and fail exactly as before. The buffer now arrives tightly sized, which also makes clip's copy a no-op.

- Cut crosslinkCacheBytes from 256 to 128 MiB. Each byte of that budget costs roughly five bytes of peak RSS -- GOGC doubles it into the heap goal, and the scavenger is paced off GC cycles, of which a faster sweep runs fewer. At 256 MiB the first two changes gave 205.7 s but 1473 MB, faster at the cost of memory. The measured curve (32/64/128/256 MiB: 369.6 s/318 MB, 292.3 s/447 MB, 249.4 s/812 MB, 205.7 s/1473 MB) puts 128 MiB where both axes improve.

Result on the same corpus: 245.4 s at 780 MB, from 293-302 s at 1251-1287 MB -- 17% less wall time and 39% less peak RSS, with instructions retired down from 3.37T to 2.48T. The gap to canonical narrows from 14.1x/6.3x to 11.8x/3.8x.

Derived tables are unchanged against an unmodified build cloning the same server: blob 67530, event 27099 (identical per type), mlink 104919, plink 23741, tagxref 51766, leaf 1426, filename 1496. The corpus parity gate passes.

What remains is structural rather than a missing optimization: a clone expands every blob twice, once in the receive-time phantom-fill cascade and once in the final whole-repository sweep, where canonical crosslinks each artifact from content it already holds in hand at receive time. Cache.Expand splits 45.4 s / 35.3 s across those two passes. Collapsing them is a change to the receive path and belongs in its own issue. check-in: 8b16c91f5c user: [email protected] tags: trunk

A full clone of the Fossil self-hosting repository disagreed with what canonical fossil derives from the SAME artifacts: mlink -3,351 rows, tagxref -632, plink -5, leaf -1. `fossil rebuild` on our own clone put every one of them back, so the artifact set was complete and the fault was entirely in crosslink-time derivation.

Six independent causes, each verified against `fossil rebuild` on a 67.6k-blob corpus clone:

mlink, -3,351. An mlink row records a parent-to-child TRANSITION, and a merge check-in has more than one parent. Canonical runs the same file-level diff once per parent -- primary, then each merge parent, then each cherrypick source -- and flags the extra rows isaux=1; we only ever diffed against the primary parent. add_mlink is now ported whole (internal/manifest/addmlink.go), which also brings the two passes the old code approximated: the delta-vs-delta revert/resurrect pass, and the baseline-child deletion pass with its mperm=0 delete rows. "Added by merge" (pid=-1) follows canonical's rule too -- a count over the rows a file produced, not a name lookup in the merge parents. A parent whose manifest cannot be read now yields NO rows rather than a whole tree of bogus "added" rows, and the rows are derived later from the parent's side once it arrives (linkPlinkChildren), so the result no longer depends on the order the sweep visits artifacts in.

tagxref, -612 of -632. A check-in's T-card does not have to name the check-in carrying it: `fossil merge --integrate` records the branch it closed as `T +closed <hash>` on the merge check-in itself. Canonical applies a check-in's, a control artifact's and a technote's T-cards through one code path; we applied only the self-referential ones on check-ins and none at all on technotes.

tagxref, the rest. A wiki-

or event-<id> tag's value is the W-card body measured after LEADING whitespace is skipped, not the raw card length.

Artifact identification. The sweep parses every blob in the repository, so what counts as an artifact decides which rows exist at all. Three gaps: a bare card letter with no arguments is legal (fossil's own first check-in spells its parentless-ness as a bare "P" card) and rejecting it dropped that check-in entirely; an N-card after a P-card is legal in a technote for historical compatibility, and rejecting it dropped real technote revisions with their event-<id> and bgcolor tags; and the check-in-versus-control decision turns on a comment, files or parents, which made `merge --integrate` check-ins read as control artifacts. Fossil's required-card table now gates linking (isArtifact), on both the sweep and the dephantomize cascade, so a file whose first line parses as a Z-card no longer links as a dateless check-in and a wiki revision missing its U-card no longer links at all.

plink, -5. The ancestry a check-in declares is a property of the check-in, not of what happens to be in the repository when it is crosslinked. A parent hash with no blob yet now reserves a phantom the way canonical's uuid_to_rid does, so the edge exists whether the parent arrives in a later round or never; skipping it lost the edge for good, because nothing revisits an already-linked artifact.

leaf, -1. Nothing calls leaf_rebuild() outside `fossil leaves --recompute`; the table is maintained by leaf_check() over the rids leaf_eventually_check() schedules -- each rid a branch tag lands on, plus that rid's parents. Drawing candidates from event instead both missed phantom parents (which fossil does list) and added check-ins that carry no branch tag and no ancestry (which it does not).

After: a fresh corpus clone and `fossil rebuild` of that clone agree row for row on mlink, plink, tagxref, leaf AND event -- 104,919 / 23,741 / 51,766 / 1,426 / 27,099 either way, and `fossil test-integrity` reports 0 errors.

Not addressed here: the local commit path (manifest.insertMlinks) still writes one mlink row per file rather than per change, and does not record merge-parent transitions. That is a separate divergence from the clone parity this issue is about.

Co-authored-by: Dan Mestas <[email protected]>

A pull into an already-up-to-date clone of the Fossil SCM repository took 2m44s and moved nothing. A CPU profile of that no-op pull puts 134.4s of 149.1s of samples (90%) inside manifest.CrosslinkContext, the whole-repository sweep Sync runs after the round loop. Nothing in the protocol path registers.

Two independent costs, both O(repository size) and neither bounded by what arrived:

collectCrosslinkCandidates 99.5s (67%) linkBatch -> Cache.Expand 34.9s (23%) of which hash.SHA3 26.0s (17%)

The query was quadratic. Its four exclusions were correlated NOT EXISTS subqueries, and tagxref is indexed on (rid, tagid) and (tagid, mtime) -- neither covers srcid -- so SQLite rescanned all 51,134 tagxref rows once per blob, 67,615 times. EXPLAIN QUERY PLAN showed the CORRELATED SCALAR SUBQUERY -> SCAN tx. Rewritten as NOT IN, each set materializes once and is probed: identical 39,727 rows, 47.7s -> 0.08s measured under canonical fossil's SQLite.

The candidate set never shrank. A blob qualifies when it has none of the derived rows a crosslink writes, and ordinary file content never gains those rows however often it is examined -- so 39,727 of the repository's 67,615 blobs were re-expanded (delta chain walked, SHA3 re-verified) and re-parsed on every sync. A probe run confirms the sweep linked exactly 0 artifacts on each of two consecutive runs over the converged repository, at 3m20s and 3m15s.

Canonical fossil has no equivalent cost because it never sweeps: xfer.c attempts one crosslink per blob as it arrives, artifact or not. This gives the sweep the same once-per-blob property via crosslink_nonartifact, an on-demand table recording blobs that expanded but hold no parseable artifact. Both facts are properties of immutable blob content, so the verdict cannot go stale; a blob that failed to expand, or that parsed but was deferred, is deliberately not recorded and is reconsidered next sweep.

Measured on a clone of the Fossil SCM repository (67,615 blobs), no-op pull:

before 2m43.9s after 0.40s

Clone is unchanged at 5m30s (it pays the one-time examination either way), and the two clones' derived tables are identical -- blob, event, mlink, plink, tagxref, filename and leaf totals all match row for row.

Fixes #202 check-in: 3748b95de7 user: [email protected] tags: trunk

* fix: make authenticated sync interoperate with canonical fossil (#203)

Every credentialed sync against a real fossil server was answered "login failed", whether or not the password was right. Push always needs a login, so push was unusable against any real server, as was pull from anything that did not grant anonymous read.

The login algorithm was never the problem. Two wire-level defects were, and each on its own produces that identical, password-independent rejection.

The push and pull cards carried their two codes in the wrong order. The canonical form is `push SERVERCODE PROJECTCODE` — canonical's own client writes `blob_appendf(&send, "push %s %s\n", zSCode, zPCode)` and both ends read the project code out of the second argument (src/xfer.c). We emitted and parsed project code first. A clone therefore adopted the server's server-code as its project code, and since a login signature is salted with the project code, every signature was derived from the wrong value. The same inversion put the server code where a server validates the project code, so an authenticated push was refused with "wrong project" even once the login worked.

The nonce covered a random comment that was never transmitted. The nonce is the hash of every byte following the login card, and the server re-hashes those bytes to check it (src/xfer.c check_tail_hash) — so the comment supplying the randomness has to be on the wire, not merely in the hash. We appended it to the hashed payload only, producing a nonce the server could not reproduce; it rejected the login before comparing the password at all, which is why a wrong password looked exactly like a right one. Comment lines now have a card type, so they can be sent as well as hashed.

Two smaller faults on the same path: a clone treated the server's round-0 "not authorized to clone" as fatal, ending the session before credentials were ever offered, where canonical tolerates that error on the first cycle precisely so the client can retry with the login card it can only build once it knows the project code; and the sync path signed with SyncOpts.ProjectCode rather than the code it had already resolved from the repository, so a caller that let the repo supply its project code — as the push and pull cards allow — panicked instead of authenticating.

The password derivation itself, sha1("<project-code>/<login>/<password>"), was correct and is unchanged. It is now pinned two ways: against the hash the fossil binary actually stores in user.pw, and against a login card captured from fossil 2.28 on the wire. Canonical's rule that an already hashed 40-hex password is used as-is is now honoured too.

Regression coverage stands up a canonical fossil server with anonymous capabilities removed, so a login card is the only way in, and asserts that a credentialed clone transfers content, a wrong password is refused, and a local commit pushed under authentication lands in the server's blob table and on its timeline. Nothing in the suite authenticated against real fossil before; clone was only ever exercised against servers granting anonymous read, where the login card is never sent, which is how this shipped.

Four xfer codec expectations that pinned the inverted argument order are corrected to the canonical bytes.

* fix(sync): correct the nonce doc to match what the server recomputes

The comment described the nonce as covering the encoded non-login cards and the signature as using the stored password hash. Both readings are wrong in ways that matter to anyone changing this code: the nonce is defined over the bytes that actually follow the login card on the wire, because the server recomputes it from what it received (src/xfer.c check_tail_hash), and the signature is salted with the project code rather than derived from user.pw directly. Hashing bytes the message does not carry produces a nonce the peer cannot reproduce, which is exactly the password-independent rejection this fix was chasing. check-in: a745f4f2ac user: [email protected] tags: trunk

AddTag, ApplyTag and ApplyTagWithTx each carried their own copy of the same type guard and eight-argument propagate call. #198 had to apply its mtime guard to all three, and a fourth site -- or one site updated while the others are missed -- reproduces exactly the shape that produced #197 (leaf derived two different ways) and #199 (project code resolved two different ways).

Extract propagateAfterInsert so the guard travels with the propagation instead of being restated at each caller, and cover the consolidated path. check-in: 4806de7907 user: [email protected] tags: trunk

Branch tag propagation left 1,155 of 20,859 check-ins on a clone of the fossil repository carrying a different branch value than canonical, and tagxref at 68,883 rows against canonical's 51,764. Two independent defects in the port, both confirmed against fossil's own source and a canonical rebuild of the same blobs.

tag_insert is mtime-guarded and our tag application was not. Canonical opens it with

SELECT 1 FROM tagxref WHERE tagid=%d AND rid=%d AND mtime>=:mtime

and returns without touching tagxref -- and without calling tag_propagate -- when it matches (src/tag.c:173-186). AddTag, ApplyTag and ApplyTagWithTx all wrote unconditionally, so a check-in's own inline T-cards clobbered a later control artifact that had retagged the same check-in, then re-propagated the stale value to every descendant. On the real corpus one branch rename did this 19,115 times: check-in ab0a0d76 declares branch dual-license, three control artifacts later rename it to clear-title, and we reinstated dual-license and flooded the repository with it. The guard makes the outcome independent of the order the sweep visits the two artifacts in.

repairTagPropagation seeded from the wrong node. tag_propagate's per-child test is strict --

coalesce(srcid=0 AND tagxref.mtime<:mtime, 1) AS doit

-- so a descendant that already carries the tag holds it at exactly the origin's mtime, doit is 0, and the walk neither retags that child nor queues it. Replaying from tag origins therefore stopped dead at the first already-tagged descendant, stranding any check-in linked by a later sweep. Canonical seeds from the primary parent instead, right after inserting a new check-in's plink edges (src/manifest.c:2300-2302 and 2467-2469), which only ever tests that parent's own children. Seed from every primary parent, which is the sweep-order-independent equivalent. Worth 693 missing propagated branch rows.

Measured on a clone of the fossil repository against a canonical rebuild of the same blobs:

divergent branch values 1,155 -> 1 (of 20,858 compared) tagxref rows 68,883 -> 51,132 (canonical 51,764) leaf rows 1,463 -> 1,425 (canonical 1,426)

The residual 632-row tagxref gap is 613 missing `closed` singletons plus small change; it measures identically before and after this change and is a separate defect. The one remaining divergent branch value is a NULL-vs-empty-string difference in cancel-tag values, also pre-existing.

repairTagPropagation still runs before repairLeafTable, as #197 requires. check-in: 7043d6e3a9 user: [email protected] tags: trunk

test.yml had zero references to wasm or GOOS=js, so the WebAssembly build was never compiled on any leg. Browser support is the reason the ncruces driver is in the tree, and a break there would have shipped undetected -- including in db/config_wasm.go, whose js-tagged wasmPragmaOverrides() no job compiled at all.

Adds a build-only job covering 31 of the main module's 33 js-resolvable packages plus the ncruces driver submodule. The package set comes from go list rather than a hand-written list so new packages are gated by default. Two main packages are excluded, both for the same structural reason: cmd/libfossil and examples/embed-repo-api import the modernc driver directly in their own main.go, and modernc.org/libc has no js/wasm build. internal/stash and internal/undo declare //go:build !js in source and are dropped by go list itself. Building under -tags test_ncruces keeps internal/testdriver in the gate by pointing it at the driver that can actually target WASM.

Running tests under WASM needs a browser harness (wasmbrowsertest) and is deliberately out of scope; compilation catches the realistic failure mode. check-in: 10abca922c user: [email protected] tags: trunk

* fix(sync): derive the project code from the repo instead of panicking (#199)

Supplying User/Password made SyncOpts.ProjectCode mandatory in practice: buildLoginCard passed the caller's raw value straight to computeLogin, which asserts it is non-empty, so an authenticated Sync on a repo whose project code the caller had not repeated took the process down. Nothing in the public API signalled that a password made the field required, and a server embedding the library died with it.

buildRequest already fell back to the repo's stored project-code for the push/pull cards; that fallback was local to one function, which is exactly how the login path drifted out of step with it. Resolve the code once when the session is built — caller value first, the repo's own config second — and let buildRequest and buildLoginCard both read it. A code that genuinely cannot be determined now returns a descriptive error from the site that needs it rather than panicking.

Sync's deferred result-not-nil assertion fired during panic unwinding, where err and result are both nil, replacing the original cause with its own message. The invariant only holds on a normal return, so it no longer asserts while a panic is in flight.

* fix(sync): write received artifacts' visibility inside the storing transaction (#200)

Receiving an artifact cost three write transactions, not one. The blob was stored in a transaction, its dephantomize cascade committed another, and then applyPrivateStatus re-derived the rid on the pool and issued a bare DELETE FROM private -- an implicit transaction of its own, with its own BEGIN IMMEDIATE and fsync, for a row that a transaction microseconds earlier had guaranteed did not exist. Every igot card announcing an artifact already held cost one more. Measured against a real fossil peer: 1509 statements executed outside a transaction to receive 1504 artifacts.

That is minutes of pure commit overhead on a large repository -- roughly 200,000 write transactions for a 67k-artifact pull -- and it is fragile as well as slow. Each of those writes contests the database's single write lock independently, so N artifacts means N chances to lose that contest and fail outright rather than one chance to wait at a single BEGIN.

Visibility now travels with the content it describes: storeReceivedFile takes the visibility a private card announced and records it inside the transaction that stores the blob, using the rid that transaction already has. The status is now atomic with the content rather than a separate write that could be interrupted between the two. The igot path buffers its updates and writes them once per round. The same three-transaction shape on the server's receive path goes with it.

The same real-fossil pull now executes 2 statements outside a transaction for those 1504 artifacts, down from 1509. db.DB.PoolExecs makes that budget assertable, which is how the regression test pins O(rounds) rather than O(artifacts) without needing a large corpus.

On the failure originally reported: the "database is locked" error came from a harness that copied a live SQLite database out from under a running server without its WAL, so that specific lock holder is not a product defect and this does not claim to fix it. The transaction storm above is a real performance and robustness defect on its own evidence, and is what this changes.

* test(sync): pin the transaction budget, not just writes outside a transaction

The budget tests measured PoolExecs, which counts statements issued outside a transaction. That pins one half of the defect and misses the other: moving a per-artifact write from the pool into its own WithTx leaves PoolExecs unchanged while every artifact still pays a BEGIN IMMEDIATE and an fsync, and still contests the single write lock on its own. That is the cost issue #200 actually measured -- roughly three write transactions per received file, about 203,000 for one clone-sized pull.

Verified by mutation: reverting the batched visibility flush to one transaction per update left both budget tests green. They now count transactions too, via a new db.DB.WriteTxns counter, and the same mutation fails with "began 51 write transactions for 50 igot cards over 1 rounds; want <= 4".

Also stop the contention test from starving the pull it is meant to observe. The rival connection re-acquired the write lock immediately on release, so on a loaded machine it could hold the pull's single BEGIN past busy_timeout and fail the test even when the batching under test is perfect -- which is what happened on CI. It now yields between bursts, modelling another process doing occasional writes rather than testing SQLite's fairness. check-in: 18bfbc166c user: [email protected] tags: trunk

* fix: derive leaf from fossil's branch rule and bound the deferral rollup

Leaf derivation dropped 66% of rows on the real Fossil repository (487 of 1,426) because both leaf paths treated any parent link as retiring a leaf. Canonical fossil retires a check-in only when a child carries the *same* branch name, with an absent branch tag reading as "trunk" (src/leaf.c: is_a_leaf / leaf_check / leaf_rebuild all share that predicate and none consult plink.isprim). Verified against fossil 2.28 itself: after a branch commit off the trunk tip, both the incremental table and `fossil rebuild` keep the trunk tip as a leaf even though its only child is its primary child, and a branch tip merged into trunk stays a leaf as well.

Two code paths were wrong, in the same way:

- repairLeafTable excluded every `pid` in plink, so merged branch tips disappeared -- the class that accounts for 939 of the corpus's leaves. - Checkin deleted every parent from leaf, so committing onto a new branch silently retired the branch point.

Both now share leafCheck/repairLeafTable in internal/manifest/leaf.go. The recompute also has to run after tag propagation rather than before it: a check-in that declares no branch of its own only has a branch value once propagation has written it, and comparing against a missing value made every inherited-branch tip look like a leaf.

Applying the new predicate to the real repository's own event/plink/tagxref yields 1,428 against fossil's 1,426, and every one of the six differing rows is fossil's incremental bookkeeping rather than its predicate: 2 are phantom rids fossil's leaf_check inserts as plink parents of check-ins it linked, and 4 are parentless check-ins with no branch tag, which nothing ever enqueues for a leaf check. Running fossil's own leaf_rebuild() on that repository gives 1,494, so it is leaf_check, not leaf_rebuild, that the shipped table reflects.

Also caps the deferral rollup's identifier lists (issue #194). A clone deferring 180 check-ins on 417 missing blobs rendered a single ~28 KB log line; the counts stay exact at INFO and the lists print eight entries plus an "...and N more" so nothing is dropped without saying so.

Fixes #189 Fixes #194

* chore: retrigger code scanning after branch restore check-in: df495ba69e user: [email protected] tags: trunk

A clone of a large repository is a conversation of dozens of rounds spanning minutes. A single transient disconnect anywhere in it aborted the whole operation with "http transport: Post ...: EOF" and produced no store at all: there was no retry, no reconnect, and minutes of transfer were discarded for one dropped packet. Reported failures landed at rounds 6, 11 and 18 across otherwise identical runs, which is the signature of independent random drops rather than a deterministic defect.

Both round loops now send a failed round again after a bounded exponential backoff — four attempts, 250 ms doubling to a 2 s cap — but only when the failure was the network dropping the exchange. The classification is an allow-list, not a deny-list: a reply that arrived whole and would not decode, a card the peer rejected, a 4xx, or an unrecognised error from a Transport this package does not know all still fail on their first occurrence. Retrying those would turn a reproducible defect into an intermittent one, which is worse than the failure retrying exists to prevent.

Telling the two apart cannot be done reliably after the fact — a truncated zlib body and a truncated wire read both surface as io.ErrUnexpectedEOF — so the transports tag their own wire stages via the optional TransportFault interface, in the same shape as FramedTransport. Non-2xx replies are now reported as such instead of being handed to the decoder, which also stops a peer's outage from being misreported as a framing defect.

The caller's context governs throughout: a cancellation or deadline aborts immediately, including during a backoff wait, so retrying cannot extend an operation past the bound its caller set (#166, #167, #183).

Resuming a partially-completed clone across process exit is deliberately not included: it needs the clone cursor persisted, a staleness policy for the server's snapshot bound, and a separate entry point, and within-session retry already preserves the expensive part. check-in: b2928e9629 user: [email protected] tags: trunk

All three submodules required the root module at v0.6.4-0.20260724230440-48111c1c984d. That pseudo-version names a commit pushed on the #118 PR branch, which was squash-merged — rewriting the work as a new commit and orphaning 48111c1. `git branch -r --contains` finds it on no branch; it survives only in proxy.golang.org's cache.

CI never noticed because it resolves through the proxy. Direct-from-git resolution fails outright:

$ GOMODCACHE=$(mktemp -d) GOWORK=off go mod download github.com/danmestas/go-libfossil go: github.com/danmestas/[email protected]: invalid version: unknown revision 48111c1c984d

v0.7.0 is tagged and reachable, so the bootstrap constraint that forced a pseudo-version is gone. Repoint all three at the tag and tidy. The root go.mod hash is unchanged between the orphaned commit and v0.7.0, so this is a pure provenance fix with no dependency movement.

Add a release guard alongside the existing `replace` guard: any in-repo module required at a pseudo-version now fails the release before the tag ships. The pattern anchors on the 14-digit timestamp and 12-hex hash, and accepts either separator before the timestamp so the vX.Y.Z-0.<ts>-<hash> form is caught as well as v0.0.0-<ts>-<hash>. Third-party pseudo-versions such as ncruces/go-sqlite3-wasm are deliberately untouched. check-in: 9d35407ecb user: [email protected] tags: trunk

* fix: write the ticket event row with its tag (#184), quiet the cascade (#186)

crosslinkTicket applied the tkt-<uuid> tag and returned the event row as a pending item for a second pass that was a no-op stub, so the row was never written at all. collectCrosslinkCandidates excludes any blob that already has a tagxref row naming its own rid as srcid, so the tag the incomplete operation wrote also hid the artifact from every later sweep. Cloning fossil's own repository produced 96 event rows of type 't' against canonical fossil's 3,808 -- 100% of the D,J,K,U,Z ticket-change shape lost, with nothing logged and test-integrity still clean, because only derived tables were wrong.

Implement the ticket rebuild instead of reordering the writes: there was no event write to reorder. ticketRebuildEntry replays every artifact carrying the ticket's tag in mtime order and derives its ticket, ticketchng and event rows, mirroring fossil's ticket_rebuild_entry, ticket_insert and manifest_ticket_event. It runs inline in the transaction that writes the tag, so the marker meaning "already linked" and the rows that make that true commit together or not at all. Replaying the whole tag set per artifact also makes the result independent of arrival order.

The event wording, the ticket/ticketchng column mapping including '+field' append semantics, the %h escaping and the two hyperlink hash lengths were all derived from the fossil binary rather than assumed, and are pinned by TestFossilBinaryTicketParity: a corpus built with `fossil ticket add`/`change` is crosslinked by us, rebuilt by fossil, and the two derivations compared row for row. TestCrosslink_TicketArtifactsProduceEventRows is the count assertion whose absence let this ship.

Audited the other three exclusion predicates for the same split-write shape: forumpost and attachment are each written by the same function, in the same transaction, as the event row that accompanies them, and propagate errors, so neither can diverge. No change made to them.

Secondary (#186): AfterDephantomize warned once per blob that failed to parse as a manifest, which is the ordinary outcome for file content and what the whole-repository sweep has always skipped silently -- ~39k lines on a fossil clone, burying the failures that matter. It now skips them the way linkBatch does. A blob that parses but fails to link is still a real fault and still warns.

* docs: record the ticket rebuild's measured cost and why it stays inline

The inline rebuild is quadratic in one ticket's artifact count. The cheaper deferred shape stays atomic but needs two call sites to remember to invoke it, and a forgotten call reintroduces #184 silently. Write the measurements and that reasoning down at the function, so a later profile of a repository with one hot ticket does not turn into an optimization back into the bug. check-in: 39f646e0ae user: [email protected] tags: trunk

The per-package parity tests compare our derivation against fossil's for fixtures built in-test, so they only cover the shapes someone thought to build. Issue #184 -- a silent 100% loss of the ticket-change artifact shape (D,J,K,U,Z) -- passed that suite throughout and was found by an external benchmark cloning the real Fossil repository.

Add TestCloneCorpusParity, which builds a repository with the canonical binary, serves it over loopback, clones it with both fossil and libfossil.Clone, and compares what each derived.

The comparison is per-event-type, not just totals. On the real Fossil repository the totals read 23,396 vs 27,099 -- a 14% shortfall that reads like rounding -- while the per-type breakdown put ticket events at 96 vs 3,808. Totals for blob, event, mlink, plink, tagxref and leaf are compared too, with deltas reported in both directions: our tagxref ran +17,119 over fossil's there, which is what writing a tag without finishing the work it marks looks like.

The corpus is built by the canonical binary at test time rather than downloaded -- seconds and under a megabyte, against the ~10 minutes and ~1 GB the real repository costs (#187) -- and covers check-ins with a branch and a merge, ticket creations and updates, a wiki page, a technote, an attachment, and a tag control artifact. requireCorpusShapes classifies every blob by its card set and fails if a shape falls below its floor, so a corpus that quietly stopped producing tickets cannot let the gate pass while covering nothing.

Gated behind LIBFOSSIL_CORPUS_PARITY=1 in addition to the existing fossil binary requirement, so the default suite is unchanged in runtime and outcome.

The gate fails on this commit, which is correct: #184 is unfixed, and it also surfaces a second divergence where a merged branch tip is missing from leaf.

Forum posts are not covered -- fossil exposes no CLI to create them. check-in: d01b94f4c1 user: [email protected] tags: trunk

The whole-repository crosslink sweep holds back a checkin whose referenced blobs (F-cards or B-card baseline) have not arrived locally, via deferCheckin, so a later sweep can rediscover and complete it once the blob shows up. The mid-round phantom-fill cascade (afterDephantomize) had no equivalent guard and linked such a checkin unconditionally, writing its event row despite insertCheckinMlinks silently skipping the missing-blob F-card.

Because collectCrosslinkCandidates only reselects a rid with no event row, that half-linked checkin was permanently stranded: no future sweep, no matter how many sync rounds ran or when the missing blob eventually arrived, would ever revisit it. A clone that terminated before the blob arrived could leave the repository with a leaf checkin selected by CalcUpdateVersion whose Checkout.Update would fail with "blob not found" -- with no recovery path.

Factor the sweep's deferral bookkeeping (availability cache, deferred rids, missing blobs) into checkinDeferralGuard, shared by both linkState (the sweep) and cascadeLinker (the cascade). The cascade now checks shouldDefer before opening a checkin's savepoint, so a deferred checkin writes nothing to the batch -- matching the sweep's defer semantics exactly, and preserving the invariant both paths depend on: a deferred rid must carry no event row, or the sweep's own candidate query can never find it again.

deferCheckin's sweep-facing behavior, and linkBatch's call site, are unchanged -- it is now a thin wrapper over the shared guard. check-in: 36d61bd582 user: [email protected] tags: trunk

TestChainBoundBitesAtSameLength inserted its fixture a statement at a time: one blob row per chain node and one delta row per edge, across two arms sized maxDeltaChainDepth and maxDeltaChainDepth+1. At the current constant that is ~65k statements, each in its own implicit transaction and each paying a commit.

The chain is incidental setup; the subject of the test is the bound. But the setup cost dominated the runtime and was sensitive to how loaded the machine was, so on a contended runner it consumed the package's 120s budget and failed a PR that touched only CI YAML.

Wrap both loops in a single db.WithTx so the chain commits once. This follows buildBackwardChain in chain_depth_bench_test.go, which already builds its chain this way in the same package.

The assertions are untouched, byte for byte: the two walks must still agree at maxDeltaChainDepth and disagree one past it. Verified by mutation -- widening one walk's loop bound by a single node still fails the test with the same diagnostic it was written to produce.

Measured on this machine, three runs each, whole-package wall clock:

modernc before 19.7s / 25.0s / 7.1s after 1.19s / 1.36s / 0.80s ncruces before 3.5s / 10.0s / 7.5s after 1.27s / 1.59s / 1.13s

The spread collapsing matters more than the median: the old fixture's cost tracked machine load, which is what made it a timeout risk rather than merely slow.

Refs #179 check-in: 39f9ea4d8a user: [email protected] tags: trunk

- Convert [Unreleased] to [0.7.0] and complete with remaining 67 commits - Add sections for v0.2.0 through v0.6.3 with commit-derived summaries - Add empty [Unreleased] section for future work - Highlight module path breaking change (libfossil → go-libfossil) - Maintain Keep a Changelog + SemVer format and existing prose style check-in: 35b79c90c6 user: [email protected] tags: trunk

emitMatches walked a hash bucket's entire collision chain at every target position with no limit. rollingHash is a weak multiply-accumulate over a 16-byte window, so low-entropy content — repeated tokens, runs of whitespace, ordinary SQL and C boilerplate — funnels an enormous number of source windows into one bucket, making Create quadratic in target length. Measured on the issue's repro (repeated 8-byte token, one 10-byte edit): 32 KiB 0.133s, 64 KiB 0.496s, 128 KiB 1.950s, 256 KiB 7.765s — ~3.9x per doubling — against 0.013s for a 128 KiB high-entropy control.

Bound the walk at maxChainCandidates, matching the `int limit = 250` that fossil's delta_create (src/delta.c) puts on the identical loop. The same walk also computed each candidate's match length twice, once to test it against the nHash threshold and again to read the value; compute it once.

Extract the walk into bestMatch so the bound is testable on a candidate count rather than on wall-clock time, which cannot flake on loaded CI.

Delta output changes, as fossil's own limit makes it change: a capped search can settle for a shorter match, so the encoder emits more copy commands. This lands our output on canonical fossil's — 128 KiB: 266 bytes vs fossil's 260; 1 MiB: 2142 vs 2120 — where before the unbounded walk produced 44 and 50 bytes and paid quadratic time for them.

After: 128 KiB 1.950s -> 0.018s, and 1 MiB 125.036s -> 0.062s.

Verified the deltas still apply byte-for-byte, by our Apply and by canonical fossil 2.28's test-delta-apply.

Co-authored-by: Dan Mestas <[email protected]> check-in: b1b6c09b41 user: [email protected] tags: trunk, v0.7.0

Fossil keeps a per-user global config database outside any repository and creates it lazily on first use. The fossil-gated tests invoke the binary with inherited env -- nothing sets HOME -- and go test runs packages concurrently, several of which shell out to fossil. On a fresh runner that database does not exist yet, so two test processes can race to create it: one wins and the others fail with "table global_config already exists".

Create it once, single-threaded, after the install and version check and before any build or test step, so no test process is ever the one to create it. fossil version does not open that database, so the existing version check does not incidentally cover this; fossil all list does, and is silent.

The warm-up command's exit status is ignored on purpose -- it exists to touch a file, not to assert anything. The file check that follows is the part that matters: a warm-up that silently stopped creating the database would leave the race live while looking like protection, so its absence fails the job loudly.

Refs #174

Co-authored-by: Dan Mestas <[email protected]> check-in: bba767af03 user: [email protected] tags: trunk