Timeline

* fix(sync): send content only to requesters allowed to read it

The sync server answered `gimme HASH` with the artifact whether or not a pull card had been accepted, and answered `pragma uv-hash`, `uvgimme` and `uvigot` with the unversioned catalog and files for any requester, anonymous ones included.

Match fossil's page_xfer:

- gimme is answered only when a pull card was accepted, or in a clone, which fossil also treats as pulling. - pragma uv-hash sends the catalog only to a requester with Read (o, i or Admin). uv-push-ok goes only to a requester with y; other readers get uv-pull-only, as fossil sends. - uvgimme and uvigot send nothing to a requester without Read, and uvigot adopts a client's mtime only for a requester with y. - Table sync follows the same rules: xgimme needs an accepted pull and pragma xtable-hash needs Read. - pragma ci-lock takes or reports a lock only for a requester with Write, so an anonymous requester cannot learn who holds one.

Pragma handling moves into handlePragma to keep handleControlCard short.

GHSA-mvx3-6793-8x4r

* docs(changelog): note that gating ci-lock on Write departs from fossil

GHSA-mvx3-6793-8x4r

* fix(sync): follow HTTP redirects as fossil does, never HTTPS to HTTP

Both HTTP transports used net/http's default redirect policy. It follows a redirect from HTTPS to plain HTTP, so a 307 or 308 resent the request, login card included, in cleartext. It also turns a 301 or 302 into a GET without the body and stops after 10 redirects, neither of which fossil does.

PostXfer now follows redirects the way fossil's http_exchange (src/http.c) and draft §4 describe. It replaces the client's redirect policy for these requests, then:

- refuses a redirect from HTTPS to HTTP, and to any scheme but http(s); - re-POSTs the same body and Content-Type for 301, 302, 307 and 308; - strips one trailing /xfer from the new location, as fossil's loop does (the draft says "repeatedly"; the code strips once); - fails on the twentieth redirect (MAX_REDIRECTS 20).

HTTPTransport and NewHTTPTransport both use it, including with a client passed through WithHTTPClient.

GHSA-3f8x-8fr5-mr98 check-in: 3a1014cf60 user: [email protected] tags: trunk

* test(conformance): repository-state conformance suite (#262)

Check go-libfossil against draft-fossil-repo-state-00, requirement by requirement. Each derived-state case is a scripted sequence of artifacts loaded into go-libfossil and into fossil(1); the tables derived from it are compared row by row, joined by artifact hash. Every case runs through manifest.Crosslink, as one sweep and one artifact at a time, and most also run as a ".rebuild" case through verify.Rebuild. fossil's rows must equal each expectation on every run, and each case states what ignoring its requirement would change, so a case cannot pass or fail by accident.

API cases cover name resolution, leaves, the ambiguity test, branch creation and listing, and tagging, each compared with fossil's answer on the same repository. TestRepoStateCorpus compares verify.Rebuild and a Crosslink sweep with fossil rebuild on real repositories named by CONFORMANCE_REPOS.

Known failures are marked with the placeholder Gap #0 until their issues are filed, so TestCoverage refuses them for now.

* test(conformance): repository-state suite review fixes (#262)

- Unsorted T-card cases use a non-hex name, so the ordering rule, not the hex-name rule, is what rejects them. - RS-4.2-7 crosslinks every artifact a second time over the rows the first left (repostate.4.2.relink-unchanged), instead of a sweep that links nothing. - RS-7.3-2, -3 and -4 are checked through manifest.Checkin and branch.Create, which maintain leaves one check-in at a time; a crosslink sweep recomputes the whole table. - retag-branch is built so the retagged check-in's own leaf status changes; branch membership is read through BranchLeaves, and the tagxref case it replaced is reattributed to RS-5.4-5; bad-prefix also compares tagxref. - parentless-untagged follows the draft and marks go-libfossil's crosslink, which follows fossil(1), as a deviation; the unused draft field becomes fossil, checked against fossil's rows. - Every fossil(1) command, in all suites, runs with HOME, FOSSIL_HOME and XDG_CONFIG_HOME in the test's directory, so it no longer records checkouts in the user's global configuration.

* test(conformance): mark repository-state gaps with their issues (#262)

Replace the placeholder gap marks with the issues that track each fix: #278 and #285-#294. check-in: 7fa0b9b5da user: [email protected] tags: trunk

* fix(xfer): reject non-decimal card sizes and bound payload memory

A payload card's size was parsed with strconv.Atoi, so a peer sending `file HASH -1` reached readPayload's negative-size assertion and panicked the client or server decoding the message, and a signed size such as `+5` was accepted. Sizes now go through parseSize, which follows the sync draft's `decimal = 1*DIGIT` grammar and fossil's 32-bit limit and returns a decoding error for anything else. uvfile FLAGS use the same parser, so negative FLAGS are rejected as the draft requires. The assertion stays, now unreachable from peer input.

readPayload no longer allocates the declared size up front: its buffer grows with the bytes read, so a large size over a short message fails after allocating about what was sent.

A cfile payload may inflate only to its USIZE (a margin above it for a delta) and not past its 4-byte size prefix, and all cfile payloads in one message share a MaxCFileInflateBytes (1 GiB) budget, so a few megabytes of zlib cannot inflate to tens of gigabytes.

FuzzDecode exercises the decoder over both framings.

* fix(sync): refuse malformed xrow payloads and cap HTTP reply bodies

An xrow payload or xdelete PKData from a peer was decoded into a map and used as is. JSON null gave a nil map that reached coerceJSONNumbers' assertion, and a primary-key value of the wrong type or a nested value reached normalizeValue's, panicking the client or server. Both directions now decode through decodeRowJSON: one JSON object, nothing after it, and repo.ValidateRow's rules, which require scalar values and primary-key values that PKHash can normalize. A client refuses an xrow for a table it has not registered. Column types match case-insensitively, as table creation accepts them.

Both HTTP transports read replies through xfer.ReadBody, which stops at MaxBodyBytes (MaxDecompressedBytes plus framing headroom) and returns ErrBodyTooLarge, untagged for retry, instead of reading an endless body into memory.

FuzzTableSyncPayload drives the server handler and client over arbitrary payloads.

* fix(sync): check synced-table rows and identifiers before building SQL

repo.UpsertXRow wrote a row's map keys into its INSERT as column names, and the rows of an xrow payload or xdelete PKData come from the peer, so a key such as "val, mtime) VALUES(...) --" chose the statement's columns and values, and a key could select from other tables into a synced one. This held on the server for pushed rows and tombstones, and on the client for a server's rows.

A peer's row must now use exactly the table's declared columns: repo.ValidateRow refuses any other key, which also keeps a peer from setting _owner. UpsertXRow no longer trusts its caller: it reads the table's registered schema and refuses any key it does not declare, apart from _owner on an ownership-mode table. ListXRows, LookupXRowOwner and DeleteXRowByPKHash validate the table name and the column names they interpolate, so no synced-table identifier reaches SQL text unchecked.

* fix(sync): answer reqconfig only with fossil's transferable config

The sync server answered reqconfig NAME with any row of the config table, for any requester, including one with no capabilities, so project-code, server-code and every local setting could be read by anyone who could reach the sync endpoint.

handleReqConfig now follows fossil's page_xfer and configure.c. It answers only a /GROUP name, only for a requester with Read, matches the group by prefix in fossil's table order, and sends only the names fossil transfers in that group, in fossil's encoding. The user and subscriber tables also need Admin, which counts only for a verified login, and the concealed table needs RdAddr. A refused request gets no cards and no error.

The groups, their names and their capability rules live in one place, config_groups.go. A differential test answers the same requests with fossil's own server and with HandleSync on the same repository and requires identical config cards.

* test(sync): keep fossil http from leaving a backoffice process

* fix(sync): check the login nonce against the request body

The server verified a login card's signature over its nonce but never checked that the nonce was the SHA-1 of the request body after the login line, as fossil's check_tail_hash does (draft-fossil-sync-protocol-00 §6.2). A login card captured from one request therefore authenticated any other request body.

The decoder now records, for each login card, the SHA-1 of the decompressed bytes after the card's line, and VerifyLogin refuses a card whose nonce does not match it. A login card that was never decoded has no tail and is refused.

Login handling otherwise follows fossil's page_xfer and check_login: - a refused login ends the exchange with "error login failed" instead of continuing as nobody; - a second login card is refused with "multiple login cards"; - a successful login adds the user's capabilities to nobody's instead of replacing them; - nobody and anonymous log in without a password, and the reserved developer and reader logins and accounts with an empty password cannot be logged into.

Fixes GHSA-2g5p-q5f3-477p.

* fix(repo): seed nobody with fossil's default capabilities

A new repository seeded the nobody user with "cghijknorswy", so every request without a login card could push check-ins and unversioned files, write wiki pages and tickets, and held Setup. Seed the four built-in users `fossil new` creates, with fossil's capabilities: nobody "gjorz", anonymous "hz", developer "ei" and reader "kptw".

Setup and Admin now include check-in, read and clone, as they do in fossil, so the creating user can sync once given a password. A login to an account with no stored password is refused, as fossil's check_login refuses it; the creator starts with none, and the signature would otherwise be the SHA-1 of the nonce alone.

Tests that push without credentials grant nobody push explicitly. Existing repositories keep their stored capabilities.

* test(auth): reserved-login test updates the seeded developer and reader rows

New repositories now seed developer and reader as fossil does, so the test gives the existing rows a password instead of inserting them.

* docs(changelog): group the sync hardening fixes under Security

* test(sync): reqconfig tests sign their logins over the request body

The login nonce now has to cover the request body, so the reqconfig tests build their admin requests with signedRequest instead of a bare login card. check-in: 3a6126c3e9 user: [email protected] tags: trunk

The server answered a pull-only client's igot cards with gimme for artifacts it lacked. The client was not pushing, so it never sent them and kept going round after round until it hit the round cap. Fossil's page_xfer handles igot only when the client sent a push card, and does the same here: a pull-only igot just records what the client has.

The client also treated the server's gimme cards as outstanding work even when it was not pushing, so any server that did ask (an older libfossil, or another implementation) could hold a pull open. A session without push now ignores gimme cards.

Two handler tests relied on a pull card alone producing a gimme; they now send a push card. check-in: 1e3e2df2f2 user: [email protected] tags: trunk

* test(conformance): delta-format conformance suite (#262)

Check internal/delta against draft-fossil-delta-format-00, requirement by requirement: 39 requirements, 88 cases. Decoder cases (Apply and OutputSize) are byte vectors, each also given to fossil(1) through its SQL delta functions; fossil's verdict must match unless the case records why it follows the draft instead (fossil masks the high bit, does not verify the checksum, and returns the size as a signed int). Each rejection breaks one rule and keeps the trailer valid, so check order cannot hide the rule under test. Producer cases check Create against a strict oracle and against fossil's delta_apply, and fossil's delta_create against Apply.

FuzzDeltaApply compares Apply with an independent model decoder; FuzzDeltaRoundTrip checks Apply(Create(s, t)) == t. DF-3.1-6 (Create has no 2^32 bound) has no case: it needs a 4 GiB input.

* test(conformance): delta suite names the reason each rejection must give

A rejection case could pass because some other rule rejected the input. Every rejection case now names a substring of go-libfossil's error (or ErrChecksum), and the harness fails the case when the message differs.

Vectors that were rejected for the wrong reason are fixed or reattributed: copy-truncated (DF-4-2), reject-10-nul (non-empty count, so the NUL is the opcode), copy-overflow-64 (DF-4-11), and high-bit-count, which now tells a masking decoder from a conforming one (fossil accepts it). The phantom-base case uses a literal-only delta; the content-discard case checks ErrChecksum and the store-side header refusals move to DF-4.1-3. The allocation bound gets its own row (DF-6-5) and takes the least of five runs. DF-3.1-6 notes issue #279. Drops a duplicate size-query case, the sizeCase helper and the decorative fuzz guard bytes. check-in: 0629de2beb user: [email protected] tags: trunk

* test(conformance): artifact-format conformance suite (#263)

Adds the suite for draft-fossil-artifact-format-00 with errata 4.1.2, 4.2 and 4.6.4, so the parser, writer, naming, hash policy and delta manifest code are checked requirement by requirement instead of by scattered unit tests that each encode one reading of fossil.

- requirements/artifact.tsv: 101 requirements from Sections 3 to 8, pinned to the draft and the three errata. - Parse cases are a known-good base artifact per type plus one edit, with the Z card recomputed, so the edit alone explains a rejection. Every case also runs through fossil test-parse-manifest; fossil 2.28 agrees with all of them. Card legality, duplicates, arity and hash positions are generated from the bases and the draft's own tables. - Accepted inputs must re-encode byte for byte unless marked non-canonical. - API cases cover naming, verification, the comparator, the writer, the R card, the hash policy and delta-manifest enumeration. - FuzzArtifactRoundTrip checks parse/marshal/parse stability, and TestArtifactCorpus (CONFORMANCE_REPOS) parses every artifact fossil crosslinked in real repositories.

267 cases go-libfossil fails are marked with a "#0" placeholder in artifact_gaps_test.go, grouped by the code at fault, for issues to be filed against.

* test(conformance): tighten the artifact suite after review (#263)

A rejection only counts when the rule the case names makes it, so every rejection case that is not about the Z card must now pass deck.VerifyZ first. Inputs go-libfossil rejected only through the Z check (framing, residue, sterilizer, unstripped wrappers, an extra Z token) become gaps, along with VerifyZ itself, which rejects an absent Z card and reads the last byte where Section 6.2 says it must not.

- Cases can name the substring fossil's rejection reason must contain; the unknown-letter cases sit in sorted position, the sterilizer cases are filed under card order, which is what fossil rejects them for, and generated legality cases take their requirement from the draft's table. - The ratchet that never persists the hash policy is a gap: fossil opening the repository still reads auto. - New rows for 4.7.9, 6.2 (absent Z, last byte), 6.3.4 and 7.1. - The audit refuses the #0 placeholder, a test keeps the gap map and the fuzz test's forgiven losses tied to live gap cases, and one Z-card helper replaces two.

* test(conformance): mark artifact gaps with their issues (#263)

Replace the placeholder gap marks with the issues that track each fix (#265-#275), and keep coverage_test.go within 100 columns. check-in: d52a6fe050 user: [email protected] tags: trunk

Adds the conformance package that the per-draft suites build on. Nothing in the repository has yet checked go-libfossil against the Fossil drafts requirement by requirement.

- A Case names its requirement. A Gap (an issue) or Deviation (a reason) marks a known difference. A marked case still runs: while it fails it is skipped, and once it conforms it fails until the mark is removed, so the list of known failures cannot go stale. - Requirements tables pin the SHA-256 of the drafts and errata they were written against. Those are copied unmodified into conformance/rfcs by scripts/refresh-conformance-rfcs.sh. A changed draft fails the coverage test until its table has been re-read and its pin moved. - Coverage is derived from the cases rather than recorded: - COVERED: every case conforms. - PARTIAL: the table's row says what its cases leave out. - DEVIATION: a case is marked as a deliberate difference. - FAILING: a case is marked with an open gap. - GAP: no case yet.

CONFORMANCE_REPORT=<path> writes the requirement-by-requirement report as Markdown. - fossil(1) is run on every case's input, through test-parse-manifest, so a case's expectation is checked against fossil each time the suite runs. withZ appends the Z card, so a case that edits one card never also breaks the checksum.

The harness's own rules are tested in harness_self_test.go. No suites are registered yet; the artifact-format suite is next (#263). check-in: 57a558fcbf user: [email protected] tags: trunk

* fix(manifest): store check-in, link and tag rows as fossil does (#258)

A rebuilt or locally written repository differed from fossil's in three derived values: event and plink times in their last digits, event.omtime left NULL, and a valueless tag stored as '' instead of NULL.

Fossil stores a tag time as SQLite's julianday() bound as a double, but writes event, plink, attachment and forumpost times into SQL text with its own printf("%.17g"), which emits 16 digits in long double arithmetic. TimeToJulian is now julianday() exactly (whole milliseconds over 86400000) and TimeToTextJulian ports fossil's printf conversion, so both match fossil bit for bit. The conversion follows builds where long double is a double (macOS, Windows); fossil itself differs across platforms in the last digit.

Check-in events record omtime, tag writes bind NULL for an empty value, the readers of event and plink times under the ncruces driver rebuild the text form, and the CLI's tag listing reads NULL values.

* test(db): sweep both stored julian forms; scan the text form from driver times (#258)

* test(verify): compare text times bit for bit only where fossil computes them in double (#258) check-in: 2c2e5e74b6 user: [email protected] tags: trunk