Timeline

Update now follows the target's renames and saves undo state. A file outside the cone stays a record-only step: never written, chmodded, backed up or read for undo. A local edit that a rename would carry out of the cone is refused, naming the cone. check-in: 74aa290eda user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # internal/undo/undo.go check-in: d6ac19e580 user: [email protected] tags: feat/submanifest-reader, main, trunk

Stash save now scans the checkout first, as fossil's stash_create does (vfile_check_signature), so an edit no scan had recorded is stashed too. Save takes the scanner, a *checkout.Checkout, as its first argument.

Undo and redo no longer print to standard output: they return the files they changed, as []undo.Change, in the undo table's row order, the order fossil undoes them in.

Tracked in #448 check-in: 2dfae63b63 user: [email protected] tags: feat/submanifest-reader, main, trunk

Update saved no undo state, so nothing could reverse it. It now reads the file of every step that writes or removes one before writing, as fossil's update calls undo_save, and saves those files with copies of vfile and vmerge and the checked-out version in the same transaction that moves the checkout to the target. If a write fails partway the files are put back, as fossil's undo_rollback does.

Checkout.Undo and Checkout.Redo (internal and public) swap that state back and forth and keep checkout-hash naming the restored version. The undo package gains ReadFile, SaveTx and RestoreFiles, and its undo table now has fossil's schema (isExe, unique pathname, rowid order, the executable bit read from disk), so fossil undo reverses a go-libfossil update and Checkout.Undo reverses fossil update; fossil 2.28 interop tests check both. Under js/wasm, which has no undo package, Update saves nothing.

Tracked in #446 check-in: 294a537880 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 5d0a52d706 user: [email protected] tags: feat/submanifest-reader, main, trunk

It is never checked against the user table; the check-in's rcvfrom row credits that login's uid when the table holds it and the local setup user otherwise, as fossil credits g.userUid whatever the U card says. A test pins that an unknown name commits under that name. check-in: 54c9e56786 user: [email protected] tags: feat/submanifest-reader, main, trunk

Receipt.ID took (rcvid, mtime) to identify its own row and checked it with a SELECT on every use. After a rollback the rcvid can be reused by another receipt's insert, and under a frozen clock the mtime matches too, so the receipt attached to the wrong row.

Fossil holds g.rcvid for its process and never re-reads the row (content_rcvid_init); a rollback ends the process with it. A Receipt outlives transactions, so the transaction or savepoint that makes its row now records that, forgets the id if it rolls back, and hands it to the enclosing level when released. A kept id costs no query, which takes one SELECT per stored blob off the clone path.

Also checks that a tree check-in's sub-manifests share the commit's receipt. check-in: 0269bd9461 user: [email protected] tags: feat/submanifest-reader, main, trunk

Update's rewrite meets the sparse worktree: a file outside the cone is a record-only step, counted in OutOfCone, and never planned, scanned, checked, written, backed up as NAME-original or chmodded; reloading vfile moves its record to the target. SetCone records the mtime of each file it materializes, as Extract now does. check-in: 47f89f0798 user: [email protected] tags: feat/submanifest-reader, main, trunk

Only macOS arm64 and linux-x64 fossil have been compared bit for bit, so the rebuild test no longer claims Windows is exact. julian.go says which long doubles fall back to 53 bits wrongly, and ExpandAll's doc says its errors match Cache.Expand's in kind, not always in wording. check-in: e4c84f20f9 user: [email protected] tags: feat/submanifest-reader, main, trunk

TestTextJulianDayMatchesFossil compared macOS arm64 fossil's stored values with the text form at the running platform's long double, so it failed on linux-x64, where fossil itself stores different last digits. TestTextJulianDayPerLongDouble already checks those values at 53 bits and linux-x64 fossil's at 64, so the platform-blind test goes. check-in: e499428d3b user: [email protected] tags: feat/submanifest-reader, main, trunk

checkCheckins now gathers into a staging report and copied only its issues back, so the MissingRefs count treeGap adds was lost: Verify reported a missing sub-manifest as an issue but counted zero missing references. check-in: 39b524c37c user: [email protected] tags: feat/submanifest-reader, main, trunk

The rebuild speed-up hands checkOneCheckin each check-in's expanded bytes and drops the pass-wide expansion cache. treeGap, from the sub-manifest reader, still lists a check-in's files to find a missing sub-manifest, and it needs a cache: without one every delta manifest re-expands its baseline. checkCheckins now keeps a small cache for those manifest reads only. check-in: 1a45812566 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: e3c81fb936 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md # internal/checkout/vfile.go check-in: 189088f46c user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 2a0701c7d3 user: [email protected] tags: feat/submanifest-reader, main, trunk

A public check-in could reuse content first stored privately, say by syncing a private branch, and leave it in the private table. Sync never sends private blobs to a peer without private access, so that peer got a check-in whose file stayed a phantom and could not be checked out.

storeFileBlobs now calls content.MakePublic on each file blob inside the check-in's transaction, as fossil's commit calls content_make_public for each F card (checkin.c). The blob is already queued in unsent, so the next push carries it. The sub-manifest writer already does this for trees.

Tracked in #439. check-in: 6d239246d0 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 2e1d1e6fb7 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md # internal/manifest/manifest.go # internal/sync/handler.go check-in: 6651024b58 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: c851990f41 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 92936436e5 user: [email protected] tags: feat/submanifest-reader, main, trunk

The unchanged-file pass of a commit, the on-disk half of a rename, the tracked-row lookup of Manage and Create's local records become helpers; Update's content precheck moves into buildUpdateMaps. The fossil interop test isolates fossil's home itself. check-in: 857b86be94 user: [email protected] tags: feat/submanifest-reader, main, trunk

The two drafts from zit's horizon-rfc branch at 86d4b67372, their requirements tables (later phases recorded as not yet implemented), the sparsecone suite over the cone specification and the sparse worktree, and the horizon suite's one case. zit's serialized cones, the files its sparse worktrees materialize over flat and tree histories, and a commit from a zit sparse worktree are recorded under testdata/zit-cones and replayed without zit. check-in: d9b18ab619 user: [email protected] tags: feat/submanifest-reader, main, trunk

Cone, NewCone, CheckoutCreateOpts.Cone, Checkout.Cone and SetCone, UpdateResult.OutOfCone and ErrOutsideCone; repo open --cone/--cone-not and repo cone (show, add, remove, set, dense, test). check-in: 94304a520c user: [email protected] tags: feat/submanifest-reader, main, trunk

A checkout created with a cone materializes, scans and tracks only the files it covers; vfile stays the full record, so fossil still reads the checkout. Commits carry uncovered files from the baseline by hash, update applies uncovered changes to the record only, and tracking, renaming, reverting or merging across the cone is refused naming it. SetCone widens and narrows. check-in: 3672f3d4fb user: [email protected] tags: feat/submanifest-reader, main, trunk

ListFilesFiltered narrows a check-in's file set to a filter's coverage; on a tree manifest the walk skips every sub-manifest whose key range the filter cannot cover. File.UUID carries an artifact the repository holds into a check-in by its name, verbatim. check-in: 337dde8a34 user: [email protected] tags: feat/submanifest-reader, main, trunk

Inclusion and exclusion members as literal path prefixes, normalized through tracked-path validation, longest member deciding, dead exclusions and contradictions refused, and zit's canonical serialized form (version 1, and version 2 for exclusions) with its SHA3-256 digest. check-in: 027fc6c741 user: [email protected] tags: feat/submanifest-reader, main, trunk

The post-commit re-tune re-reads the settings in its own transaction and does nothing unless the pin is still the commit's k and submanifest-auto-retune is still on, so a manual re-tune or the automatic one switched off since the commit is never overwritten. A sub-manifest a public commit makes public is queued for the next push, since it was never sent while private. The CLI states the k range from the manifest constants, and buildCheckinDeck's R-card computation moves to checkinRCard. check-in: c907b009c7 user: [email protected] tags: feat/submanifest-reader, main, trunk

Repo.EnableSubmanifests(0) analyzes the main-branch tip, and when no check-in is on the main branch (a repository whose history sits on another branch name) the newest check-in by time, the highest rid breaking a tie, rather than refusing; an empty repository still refuses. The godoc names the check-in in each case, and DisableSubmanifests' says existing trees remain and keep being served and that no note is returned.

`repo submanifests status` fails when a setting cannot be read instead of printing defaults, and shows a stored k outside 1..12 as invalid, with commits falling back to flat manifests. check-in: a659919be4 user: [email protected] tags: feat/submanifest-reader, main, trunk

`repo submanifests enable --k 0` ran automatic selection; §8.1 refuses an explicit k outside 1..12, so only an absent --k selects now. The SDK pages are rebuilt with gomarkdoc's repository flags, which a worktree cannot detect, so their source links match the published pages again. SM-7.2-3's note says the pragma is not reachable from the public API. check-in: 7a2f6dbe7a user: [email protected] tags: feat/submanifest-reader, main, trunk

- A public check-in makes every sub-manifest it names public, so a leaf a private check-in stored first is sent with the tree. - Sub-manifests are named by the hash policy alone (StoreUnderPolicy), in sha1, sha3 and auto repositories alike, and only those the commit wrote are queued for sync. - A sub-manifest that fills a phantom links the check-ins waiting on it once the commit is durable (§7.1, §7.3), not at the next sweep. - The automatic re-tune runs after the commit is durable, as zit runs it; its failure is logged and never fails the commit. - Enabling selects outside any write transaction, replaying one check-in at a time, then re-reads the settings in a short write; a pin that appeared meanwhile takes §8.1's precedence. - Checkin's transaction body moves to checkinTx, so neither function exceeds 70 lines. check-in: 0f543785e1 user: [email protected] tags: feat/submanifest-reader, main, trunk

StoreUnderPolicy names content by the repository's hash-policy algorithm and never reuses a copy held under the other algorithm's name, as draft-fossil-submanifests-00 §6.2 requires of an S card's hash. It reports whether the row was found, inserted, or filled a phantom, so a caller can queue only what it wrote and retry what waited on a phantom. Store shares its write-and-verify tail. check-in: 315b5021fc user: [email protected] tags: feat/submanifest-reader, main, trunk

Repo.EnableSubmanifests(k) now analyzes the current trunk, the check-in the next commit most plausibly descends from, instead of the newest check-in; with no trunk check-in only an explicit k succeeds. A test puts a newer one-file check-in on another branch to show which one is read. Repo.DisableSubmanifests returns only an error and documents that existing trees stay.

`libfossil repo submanifests status` prints the three settings and whether the repository holds a tree; `retune` requires --k, and `disable` prints zit's note when trees exist. The SDK pages are rebuilt with make docs-gen-sdk. check-in: 5377023003 user: [email protected] tags: feat/submanifest-reader, main, trunk

On four flat two-commit histories zit wrote, go-libfossil reproduces every manifest byte for byte and then enables at the k zit pinned; in each the replay moves the pin off the snapshot's value, so the replay port is what is being checked. The pins are recorded for CI under testdata/zit-writer, and SM-6.3-7 states the refinement. The changelog describes the API and CLI. check-in: aea1e207f4 user: [email protected] tags: feat/submanifest-reader, main, trunk

Repo.EnableSubmanifests(k) now selects over the newest check-in itself, so callers pass only k (0 to select); Repo.DisableSubmanifests joins RetuneSubmanifests. `libfossil repo submanifests enable [--k N]`, `retune --k N` and `disable` drive them, printing the pin and the operator warning. The libfossil and cli SDK reference pages are regenerated. check-in: 0ebee94560 user: [email protected] tags: feat/submanifest-reader, main, trunk

The draft's automatic selection is advisory, and zit lets history outrank the snapshot: on enable it replays the newest 200 check-ins through the tree builder at the snapshot's k and its two neighbours and pins the value whose steady-state commits materialize the fewest bytes, the lowest on a tie, keeping the snapshot when fewer than two check-ins replay or every candidate stays flat. replayK ports that step by step, so two writers enabling on one history pin alike.

DisableSubmanifests turns emission off and keeps the pin, returning zit's note when trees already exist. The out-of-range fallback logs zit's report. check-in: 6d79130418 user: [email protected] tags: feat/submanifest-reader, main, trunk

The submanifest table gains the writer's requirements: the §6.1 predicate, §6.2's construction, limits, merge, stops, canonical encoding and storage order, §6.3's pinning, out-of-range fallback and selection, §6.4's manual and automatic re-tune, §8.1's enable rules and disabling, and the writer vectors of A.5. The writer reproduces every Appendix A tree, the A.5 three-level vector at k = 2 and the height-0 merge and encoding vectors at k = 3. 71 requirements: 66 covered, 3 partial, 2 deviations, 0 failing.

TestZitWriterInterop, with zit present, has zit commit ten scenarios (the A.1 set; 40 files at k = 1, 2 and 12; 3,000 files at k = 1, 3 and 5; deep paths; names with spaces, tabs, newlines and non-ASCII, executables and a rename) and requires go-libfossil's check-ins byte-identical. zit then clones, verifies and lists go-libfossil's trees, and verifies a tree go-libfossil commits on top of zit's; the §6.3 selection matches zit's analysis on five file sets. TestZitWriterBundle replays the recorded zit manifests in CI. zitRun now returns stdout only and sets FOSSIL_HOME, and zitListing reads names holding spaces, tabs or newlines. check-in: b77d8b119e user: [email protected] tags: feat/submanifest-reader, main, trunk

Checkin now honours draft-fossil-submanifests-00 §8.1: with enable-submanifests on and a pinned submanifest-k it stores the canonical tree first, marks each sub-manifest unsent, records the seen-submanifest indicator that arms the §7.2 gate, and writes a manifest of S cards with no F card and never a B card. The setting is off by default, so every commit is byte-identical to before. A stored k outside 1..12 falls back to a flat manifest and logs it (§6.3); unchanged sub-manifests are shared by content addressing.

EnableSubmanifests is §8.1's local request: a valid pin wins and a different explicit k is refused as a re-tune; with no valid pin an explicit k is pinned, or §6.3's selection runs over the current check-in; a request that cannot be met changes neither setting. RetuneSubmanifests is the manual §6.4 re-tune. After a tree commit the §6.4 gate re-pins k while submanifest-auto-retune is on, and only advises otherwise.

A sync run reports avoid-submanifests in SyncResult, and CheckinOpts.AvoidSubmanifests keeps a commit made in that sync flat (§7.2). The settings' names and parsers move to internal/manifest, which sync now uses. Repo.EnableSubmanifests and Repo.RetuneSubmanifests expose the two requests, the first returning the operator warning. A test shows fossil 2.28 never receives a tree this writer made. check-in: 14f17e675e user: [email protected] tags: feat/submanifest-reader, main, trunk

BuildTree is the construction of draft-fossil-submanifests-00 §6.2 over a check-in's sorted file entries at partition parameter k: the §6.1 predicate (leading zero bits of SHA3-256 of the name), the one-pass scan with its MIN and MAX limits and final-group merge at every height, the non-reducing and single-group stops, and the height-16 cap. Sub-manifests come back in storage order, children before parents, named by a caller function so the commit path can store as it names.

partitionCosts is §6.3's C(k), computed with zit's arithmetic so the two writers select alike; SelectPartitionK picks the least cost and the lowest k on a tie, and retuneTarget is §6.4's gate with zit's 15% margin. Tests reproduce Appendix A.2, A.3 and A.4 by name and pin the scan's limits on synthetic counts. check-in: 47cb5afcd7 user: [email protected] tags: feat/submanifest-reader, main, trunk

draft-fossil-submanifests-00 §6.2 fixes minimal escaping for every filename a tree construction emits: exactly space, newline and backslash are escaped, every other byte raw. CanonicalEncode is that encoding; S cards and a sub-manifest's F cards now marshal with it, while a flat manifest keeps §4.3's, so no existing artifact changes name.

MarshalLeaf and MarshalInterior serialize the two sub-manifest shapes, and LeafCard gives one leaf F card's bytes for the §6.3 cost. A test reproduces the Appendix A.2 leaf byte for byte. check-in: e9ec7390c7 user: [email protected] tags: feat/submanifest-reader, main, trunk

Fossil has no plans to read sub-manifests, so the 22900 threshold is not only fossil 2.29's announcement: every later release clears it too. The fix is a version no fossil announces, agreed with zit and the draft; SM-7.2-1 now cites danmestas/zit#239, and SM-6.3-3's draft wording is tracked in danmestas/zit#240. check-in: 7408c02304 user: [email protected] tags: feat/submanifest-reader, main, trunk

- A clone to a client the 7.2 gate refuses could name a withheld tree as a delta source: walkCloneChain never asked the gate, and a refused artifact was still recorded as sent. A flat check-in stored as a delta of a tree manifest -- the normal flat-to-tree transition -- then reached a 2.28 client as a cfile it could never apply. A withheld source is now unforwardable, so its dependent goes out whole, and a refused artifact is not recorded as sent. Gimme answers and pushes already send full content, and igot only announces, so the clone was the one path. - The gate armed only once a crosslink set the seen-submanifest indicator, so a tree a sync had stored but not yet linked went out ungated. Every receive path now sets it as the artifact is stored -- checkDelta hands back the content it rebuilt, so a verified delta costs no second expansion -- and a repository an earlier version swept, which may hold trees that version filed as unparseable, arms the gate until the next sweep re-examines them. - The three sub-manifest settings are written inside one savepoint, so a database error partway leaves none of them applied. - The 22900 threshold is zit's and is also what an unreleased fossil 2.29 announces without reading trees. The constant says so, and the conformance suite records SM-7.2-1 as a deviation with a case that shows such a peer being sent a tree. check-in: 4deb9d870f user: [email protected] tags: feat/submanifest-reader, main, trunk

A sweep asked every delta manifest whether its baseline is a tree by expanding and parsing the baseline again, after missingBaseline had just done so. Record tree-ness where the baseline is first verified, and walk only a tree baseline. check-in: edefbcd26f user: [email protected] tags: feat/submanifest-reader, main, trunk

It runs zit, not fossil, each time under a HOME of its own. check-in: 6b694f0419 user: [email protected] tags: feat/submanifest-reader, main, trunk