Timeline

Save reads each changed file through vfile.Disk, so a symlink is stashed as its target's name and recorded with file_islink, and its revert makes it a link to the committed target. Apply writes a stashed symlink as a symlink and deletes a link before writing, so neither writes through one. check-in: 5be36b41a0 user: [email protected] tags: feat/submanifest-reader, main, trunk

restoreCommitted writes through vfile.Disk with the row's islink, so a repointed link is made a link to its committed target again instead of having that name written into the file it points to. check-in: 6a440bc42f user: [email protected] tags: feat/submanifest-reader, main, trunk

Move checkout's link-aware diskView into vfile.Disk, so checkout, undo and stash share one RepoFILE view: Stat, Read, Write (which deletes a link before writing, makes a symlink when allow-symlinks is on, creates the directory and sets the executable bit as file_setexe does), SetExe, MTime and Remove. The allow-symlinks setting is read by vfile.OpenDisk.

undo.Save, Undo, Redo, Swap, ReadFile and RestoreFiles now take a Disk: a symlink is read as its target's name and recorded with isLink, and is deleted and made a symlink again on restore, as undo_save and undo_one do, so Undo of an Update no longer writes through a link. check-in: 858c20561f user: [email protected] tags: feat/submanifest-reader, main, trunk

The cones merge called extractSingleFile without the islink argument the symlink fix added, so it did not build. SetCone now passes the row's islink, and a file entering the cone is written as Extract writes it: a symlink with allow-symlinks on. check-in: 36fccc1f87 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # internal/checkout/update.go check-in: 6708292b55 user: [email protected] tags: feat/submanifest-reader, main, trunk

Since the scan began recording islink and chnged 7/9, Create followed by Extract without Force failed on any repository with an "l" file: the file was missing, the scan read it as UNLINK, and the missing-file skip only covered plain edits.

Extract and Update now write a symlink entry as fossil's vfile_to_disk does: as a symlink to the blob content while allow-symlinks is on and the storage can make links, otherwise as a plain file holding the target. A symlink is deleted before a file is written over it, and setMergedMode leaves symlinks and directories alone, as file_setexe does. Extract's missing-file skip covers mode changes too, so the first Extract after Create plays fossil's open whatever the entry.

With allow-symlinks off, fossil itself reports such a file UNLINK and its checkout refuses without --force; Extract keeps that behavior, and an interop test pins it against fossil's open in both modes.

ScanENotFile no longer refuses a directory at a pending add or removal recorded with chnged=0, matching the else-if in vfile_check_signature.

Tracked in #449 check-in: 8cad365e82 user: [email protected] tags: feat/submanifest-reader, main, trunk

Update now follows the target's renames and saves undo state. A file outside the cone stays a record-only step: never written, chmodded, backed up or read for undo. A local edit that a rename would carry out of the cone is refused, naming the cone. check-in: 74aa290eda user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # internal/undo/undo.go check-in: d6ac19e580 user: [email protected] tags: feat/submanifest-reader, main, trunk

Stash save now scans the checkout first, as fossil's stash_create does (vfile_check_signature), so an edit no scan had recorded is stashed too. Save takes the scanner, a *checkout.Checkout, as its first argument.

Undo and redo no longer print to standard output: they return the files they changed, as []undo.Change, in the undo table's row order, the order fossil undoes them in.

Tracked in #448 check-in: 2dfae63b63 user: [email protected] tags: feat/submanifest-reader, main, trunk

Update saved no undo state, so nothing could reverse it. It now reads the file of every step that writes or removes one before writing, as fossil's update calls undo_save, and saves those files with copies of vfile and vmerge and the checked-out version in the same transaction that moves the checkout to the target. If a write fails partway the files are put back, as fossil's undo_rollback does.

Checkout.Undo and Checkout.Redo (internal and public) swap that state back and forth and keep checkout-hash naming the restored version. The undo package gains ReadFile, SaveTx and RestoreFiles, and its undo table now has fossil's schema (isExe, unique pathname, rowid order, the executable bit read from disk), so fossil undo reverses a go-libfossil update and Checkout.Undo reverses fossil update; fossil 2.28 interop tests check both. Under js/wasm, which has no undo package, Update saves nothing.

Tracked in #446 check-in: 294a537880 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 5d0a52d706 user: [email protected] tags: feat/submanifest-reader, main, trunk

Update matched files by name, so a local edit to a file the target renamed was left behind as an untracked file under the old name while the new name got the target's content.

path.NameChanges ports fossil's find_filename_changes with revOK: the renames along the primary path from the checkout's version up to the common ancestor and down to the target, a rename on the way up taken backwards. Update uses it as update.c does: a changed file the target renamed is matched to the target's file under its new name. A merge is written under the new name and the old name removed; when the target only renamed the file, nothing is written and the edit stays under the old name, untracked, as in fossil, which Unmerged now reports. A local rename is not followed, as in fossil.

fossil 2.28 interop tests compare both directions: a target-side rename and edit, a target-side pure rename, a local rename of a file the target edited, and a rename on the checkout's own branch followed backwards to a sibling.

Tracked in #445 check-in: c88a801cf9 user: [email protected] tags: feat/submanifest-reader, main, trunk

It is never checked against the user table; the check-in's rcvfrom row credits that login's uid when the table holds it and the local setup user otherwise, as fossil credits g.userUid whatever the U card says. A test pins that an unknown name commits under that name. check-in: 54c9e56786 user: [email protected] tags: feat/submanifest-reader, main, trunk

Receipt.ID took (rcvid, mtime) to identify its own row and checked it with a SELECT on every use. After a rollback the rcvid can be reused by another receipt's insert, and under a frozen clock the mtime matches too, so the receipt attached to the wrong row.

Fossil holds g.rcvid for its process and never re-reads the row (content_rcvid_init); a rollback ends the process with it. A Receipt outlives transactions, so the transaction or savepoint that makes its row now records that, forgets the id if it rolls back, and hands it to the enclosing level when released. A kept id costs no query, which takes one SELECT per stored blob off the clone path.

Also checks that a tree check-in's sub-manifests share the commit's receipt. check-in: 0269bd9461 user: [email protected] tags: feat/submanifest-reader, main, trunk

Update's rewrite meets the sparse worktree: a file outside the cone is a record-only step, counted in OutOfCone, and never planned, scanned, checked, written, backed up as NAME-original or chmodded; reloading vfile moves its record to the target. SetCone records the mtime of each file it materializes, as Extract now does. check-in: 47f89f0798 user: [email protected] tags: feat/submanifest-reader, main, trunk

Only macOS arm64 and linux-x64 fossil have been compared bit for bit, so the rebuild test no longer claims Windows is exact. julian.go says which long doubles fall back to 53 bits wrongly, and ExpandAll's doc says its errors match Cache.Expand's in kind, not always in wording. check-in: e4c84f20f9 user: [email protected] tags: feat/submanifest-reader, main, trunk

TestTextJulianDayMatchesFossil compared macOS arm64 fossil's stored values with the text form at the running platform's long double, so it failed on linux-x64, where fossil itself stores different last digits. TestTextJulianDayPerLongDouble already checks those values at 53 bits and linux-x64 fossil's at 64, so the platform-blind test goes. check-in: e499428d3b user: [email protected] tags: feat/submanifest-reader, main, trunk

checkCheckins now gathers into a staging report and copied only its issues back, so the MissingRefs count treeGap adds was lost: Verify reported a missing sub-manifest as an issue but counted zero missing references. check-in: 39b524c37c user: [email protected] tags: feat/submanifest-reader, main, trunk

The rebuild speed-up hands checkOneCheckin each check-in's expanded bytes and drops the pass-wide expansion cache. treeGap, from the sub-manifest reader, still lists a check-in's files to find a missing sub-manifest, and it needs a cache: without one every delta manifest re-expands its baseline. checkCheckins now keeps a small cache for those manifest reads only. check-in: 1a45812566 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: e3c81fb936 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md # internal/checkout/vfile.go check-in: 189088f46c user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 2a0701c7d3 user: [email protected] tags: feat/submanifest-reader, main, trunk

fossil's printf carries a %.17g value in C long double: a 64-bit double on macOS arm64 and Windows, 80-bit x87 on x86-64 Linux. The 16th digit, and so the event time fossil stores, differs between the two for about one time in five, which is why TestTextJulianDayMatchesFossilBinary passed on macOS and failed against CI's linux-x64 fossil. SQLite is not involved: julianday() counts integer milliseconds and 3.52.0's text-to-double conversion is integer arithmetic, neither using LONGDOUBLE_TYPE.

fossilPrintf17g now runs fossil's algorithm in a big.Float with the significand width of the platform's long double (53, 64 or 113 bits). Against all 4,007 times fossil stored in the failing CI run, 849 of them different from macOS fossil's, the 64-bit form matches every one; the 53-bit form still matches macOS fossil. The binary128 width (arm64 Linux) follows the same arithmetic but has not been compared with such a fossil. Times written on one platform and rebuilt by fossil on another may differ in the last place, as two fossils do. check-in: 387204fa55 user: [email protected] tags: feat/submanifest-reader, main, trunk

A public check-in could reuse content first stored privately, say by syncing a private branch, and leave it in the private table. Sync never sends private blobs to a peer without private access, so that peer got a check-in whose file stayed a phantom and could not be checked out.

storeFileBlobs now calls content.MakePublic on each file blob inside the check-in's transaction, as fossil's commit calls content_make_public for each F card (checkin.c). The blob is already queued in unsent, so the next push carries it. The sub-manifest writer already does this for trees.

Tracked in #439. check-in: 6d239246d0 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 2e1d1e6fb7 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md # internal/manifest/manifest.go # internal/sync/handler.go check-in: 6651024b58 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 044f6da5e0 user: [email protected] tags: feat/submanifest-reader, main, trunk

Stash and undo read, wrote and removed working files with the os package, so a MemStorage checkout and simulation tests never saw those writes, and no write recorded its mtime. They now take a *simio.Env and go through its Storage, recording each written file's mtime via vfile.WrittenMTime (moved with RecordableMTime from internal/checkout so these packages can share the same-second guard).

Stash save marks reverted rows unchanged with their written mtime, as fossil's revert does. Apply and pop add a stashed added file again, as fossil's stash_apply does, and give a restored edit's row its changed state and written mtime. Undo and redo record a written file's mtime only when the restored row's mtime matched the file when it was saved.

Tracked in #448 check-in: 41951ffbc5 user: [email protected] tags: feat/submanifest-reader, main, trunk

Read data through the ported BOM/UTF-16 conversion and line splitter, so the Update gate and the merge agree on what is text.

Tracked in #447 check-in: eb1791acd5 user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: 2f61d4898f user: [email protected] tags: feat/submanifest-reader, main, trunk

Verify and Rebuild expanded every blob one at a time through a 256 MiB LRU cache. A real repository's delta chains run hundreds of revisions deep, so evicted chains were walked again from their roots: the blob check on the Fossil SCM repository ran for over 45 minutes.

content.ExpandAll walks each delta tree from its root, as fossil's rebuild_step does, expanding every blob once from its parent's content and binding it to its name. It reports for each blob what a caching Expand of that blob alone reports, and expands blobs no sound root reaches (cycles, phantom-grounded or too-deep chains) on their own for that error. checkBlobs and Verify's checkCheckins both use it and sort their issues by rid, so reports are unchanged. check-in: 9a9bc65f92 user: [email protected] tags: feat/submanifest-reader, main, trunk

Port fossil 2.28's merge3.c and the diff it runs on (break_into_lines, break_into_tokens, longestCommonSequence, diff_step, diff_optimize, the UTF-8/UTF-16 BOM handling) so Merge and Update align, merge and mark conflicts exactly as fossil does: line-numbered markers, the token-level suggested resolution, CRLF markers, and binary input left unmerged. HasConflictMarkers follows contains_merge_marker in skipping CR runs.

A fossil-binary comparison test checks 1237 generated and hand-written cases against "fossil 3-way-merge" and UTF-16 cases against "fossil merge"; fossil commit refuses a file holding our markers.

Tracked in #447 check-in: dcec15d91f user: [email protected] tags: feat/submanifest-reader, main, trunk

# Conflicts: # CHANGELOG.md check-in: c851990f41 user: [email protected] tags: feat/submanifest-reader, main, trunk

Port the remaining branches of fossil's vfile_check_signature. A missing file is marked changed with mtime -1, a removed file is marked changed, and the permission half sets chnged 6-9 for exec-bit and symlink changes (returning to 0 when undone), honouring the allow-symlinks setting: off, a symlink is seen as what it points to; on, as itself, its content its target's name.

Status reports the mode changes as fossil's changes command labels them (executable, symlink, unexec, unlink). Commit records the permission it finds on disk, l included, with a symlink's target as its content. Where the storage has no file modes (MemStorage) or on Windows, the recorded permissions are kept, as fossil on Windows keeps them. ScanENotFile now refuses a non-file where a tracked file belongs, and merge scans with it as fossil's does. Extract still writes a missing file without Force.

Interop tests run the same edits through fossil changes and go-libfossil status on one checkout, in both orders and with allow-symlinks on and off, and compare a commit's F cards with fossil's.

Tracked in #449 check-in: 2d2e62d420 user: [email protected] tags: feat/submanifest-reader, main, trunk

fossil writes event, plink, attachment and forumpost times as 16-digit SQL text, and its bundled SQLite 3.52.0 reads that text with sqlite3AtoF, which multiplies by a 64-bit power-of-ten approximation and is not correctly rounded. TextJulianDayFromMillis parsed the text with strconv.ParseFloat, so about one time in 3,000 came out one unit in the last place away from fossil's: 13 rows of fossil's own repository. sqliteAtoF ports SQLite 3.52.0's conversion.

julianday() itself already matched; a sweep of 21,000 card times checks JulianDayFromMillis against the linked SQLite's julianday(), and a fossil-binary sweep rebuilds 4,000 control artifacts and requires every event time fossil stores. check-in: 342474c212 user: [email protected] tags: feat/submanifest-reader, main, trunk

* fix(checkout): keep uncommitted changes across Update, as fossil does

Update compared only committed versions: it wrote the target over a locally edited file, deleted an edited file the target removed, and merged the current and target check-ins with their ancestor while ignoring the working file.

It now follows fossil's update (src/update.c): scan the checkout, then for each file take the target's content when there is no local change, merge a local edit three ways (the checkout's version as base, the working file as local, the target as other) with conflicts listed in Conflicted, keep an edited file the target deletes as an untracked file, keep pending adds and removals, restore a missing file, and copy an untracked file the target adds over to NAME-original first. A dry run writes nothing. Files written with the target's content record the mtime of the write. Files are matched by name; a target-side rename is not followed.

The two tests that relied on merging two branches' committed changes now make the clashing edit locally. A fossil 2.28 interop test runs the same edits through fossil update and compares files, fossil changes and Status.

Tracked in #443

* fix(checkout): make Update safe on binary files, failures and dry runs

A binary file (a NUL byte, or a line over 32,767 bytes, as fossil's line diff refuses) is no longer line-merged. Update saves the working copy as NAME-original and writes the target, as fossil's update does; Merge keeps the checkout's copy, as fossil's merge does. Both used to splice the versions and report a clean merge. merge.IsText is fossil's test.

Update refuses before writing anything when a file it must write or delete has a directory or a read-only file in the way, moves the checkout database to the target in one transaction, and in a dry run leaves the database untouched (the scan's findings stay in memory). Every target file gets the target's executable bit, as fossil's checkout_set_all_exe does, so a bit-only change reaches the disk.

UpdateResult.Unmerged lists files left to reconcile without markers: edited files the target deletes, files added both locally and by the target, and binary conflicts. The interop test now covers a binary conflict against fossil 2.28.

Tracked in #443

* fix(checkout): check real write access before Update writes anything

The pre-write check read the owner-write bit: it missed an unwritable directory (a remove failed after a merge had been written), files owned by others, and wrongly refused for root. It now asks the storage, via an optional CheckWritable: OSStorage uses access(2) with W_OK on unix and a read-only-attribute check elsewhere (js/wasm, Windows). Every step needs its directory writable; a write or merge over an existing file needs the file writable too.

A read-only untracked file where the target adds one is backed up to NAME-original, deleted and replaced, as fossil does, instead of refused.

Merge keeps a binary file whatever its strategy, as fossil's merge does. setVVar's precondition now catches a nil *sql.DB or *sql.Tx inside the interface.

Tracked in #443 check-in: 6e6969bf97 user: [email protected] tags: main, trunk

ticketInsert assumed ticket.tkt_ctime and ticketchng.tkt_rid/tkt_user, so crosslinking fossil's own repository failed with "table ticketchng has no column named tkt_user". It now probes the schema as fossil's getAllTicketFields does and writes as ticket_insert does: optional columns only where present, a ticketchng row only when the artifact sets one of its fields or the table records the artifact or its user, shared fields the artifact leaves unset copied from the ticket row, and the "baseline for NAME" column set to the artifact that assigns NAME.

A fossil-binary oracle derives three such schemas and requires the ticket and ticketchng rows fossil rebuild writes. check-in: eb234f62ad user: [email protected] tags: feat/submanifest-reader, main, trunk