Artifact 71d857a6596e93691833c5db86c3c58c9031f80f353fca0b526efa608f797bd5:
- File
internal/checkout/extract.go
— part of check-in
[d810a8118f]
at
2026-10-18 20:01:53
on branch main
— fix: handle time exactly as fossil does (#442)
* fix(manifest): round a caller's check-in and tag time as fossil does
Fossil reads a --date-override through SQLite's strftime, which rounds the seconds to the millisecond (holding a fraction past .999 in its second) rather than truncating them. A caller time finer than a millisecond now gets the D card fossil would write for it; the default clock time is still truncated, as fossil reads its clock.
fsltype.CardTime and CardTimeNow are the one place both rules live, and the checkout commit path and branch creation take their default time through them.
* fix(repo): stamp config and user rows in seconds since 1970
Fossil writes config.mtime and user.mtime with its now() SQL function, time(0). SetConfig, SetHashPolicy and CreateUser wrote julianday('now') instead, so fossil read those rows as dating from 1970 when comparing times in a configuration merge, and SetCaps and SetPassword left the user's mtime alone where fossil's user command restamps it.
* fix(auth): read and write user.cexpire as a Julian day
Fossil keeps user.cexpire as a Julian day (src/login.c). A sync login parsed it as RFC 3339 or 'YYYY-MM-DD HH:MM:SS' text, so any value fossil had written failed to parse and the login was refused as expired. The expiry is now read through julianday(), which takes both forms, and repo invite --ttl writes a Julian day; its write error is now checked.
* fix(checkout): set file times as fossil's --setmtime does
Fossil's --setmtime gives each unedited file the time of the newest check-in, among the version and its primary ancestors, that recorded the file's content, truncated to whole seconds (mtime_of_manifest_file), and it does so for every unedited file of the version once the checkout is up to date (vfile_check_signature with CKSIG_SETMTIME). SetMTime gave every written file the target check-in's time to the millisecond, and an update left the files it did not write alone.
* fix(stash): keep and show stash and bisect times as fossil does
Fossil stores stash.ctime as julianday('now') and lists it with datetime(ctime); go-libfossil stored CURRENT_TIMESTAMP text, which the two drivers returned in different forms, and showed a fossil-written time as a bare number. bisect's list now shows datetime(event.mtime), as fossil's bisect ls does, instead of the Julian day.
* test(verify): fossil rebuild keeps the times go-libfossil stores
A repository go-libfossil wrote, with check-ins at caller and clock times, a tag and a branch, is rebuilt by the fossil binary, and every check-in event, plink and tagxref time must come back unchanged.
* test(sync): a fossil clone and push agree on every synced time
The fossil binary clones a go-libfossil server, unversioned files included, commits at a time finer than a millisecond and pushes back. Each side derives check-in event, plink and tagxref times from the artifacts itself, and the unversioned time travels on the wire; both repositories must then hold the same values.
* fix(resolve): bound NAME:TIMESTAMP with julianday() as fossil does
Fossil resolves BRANCH:TIMESTAMP with event.mtime<=julianday(BOUND). The bound was computed in the text form event rows hold, so on a millisecond where that form is larger than julianday()'s, a check-in at the bound itself resolved here and not in fossil.
* fix(manifest): round a caller's card time with integer arithmetic
CardTime followed SQLite's printing of the seconds as a double, which is not a well-defined rounding: at exact half-millisecond ties fossil 2.28's SQLite gives .124 for .1245 but .020 for .0195, and other SQLite versions differ again. A caller time is now rounded to the nearest millisecond on its nanoseconds, ties up, held at .999 as fossil holds it. The fossil binary test compares every non-tie value, across second, minute, day and year ends; ties are asserted on their own.
* fix(auth): ignore user.cexpire at sync login, as fossil does
user.cexpire is the expiry of fossil's web-login cookie: fossil sets it at web login, writes 0 at logout, and its sync login (check_login) never reads it. Refusing a sync login on it locked out anyone who had logged into or out of fossil's web UI, or whose cookie had lapsed.
The invite TTL that relied on it now lives in a config row of its own, invite-expire:LOGIN, as a UTC instant checked at sync login; repo invite no longer writes cexpire, and the expiry is no longer written in local time and read as UTC. Deleting a user drops its invite expiry.
Tracked in #441.
* fix(repo): convert Julian config and user mtimes on open
Repositories go-libfossil wrote before config and user rows were stamped in seconds since 1970 still hold Julian days there, which fossil reads as January 1970. Open converts those rows to the seconds they stand for. It checks first and writes only when a row needs it, so it converts once and takes no write lock on an up-to-date repository.
* fix(checkout): set a file no check-in recorded to 1970, as fossil does
With no mlink row for a file's artifact in the version's ancestry, fossil's max(event.mtime) is NULL, which mtime_of_manifest_file reads as 0, so --setmtime sets the file to 1970. SetMTime skipped the file.
* fix(stash): list stashes by creation time, as fossil's stash ls does
fossil stash ls orders by ctime, oldest first. List ordered by stash id, newest first; it now orders by ctime with the id breaking a tie.
* fix(auth): clear a login's invite expiry with the user, atomically
DeleteUser removed the user and its invite expiry in two statements; a failure between them left the user's expiry behind or the user without it. Both now go in one transaction. A user fossil deleted leaves its invite-expire row behind, so CreateUser clears it in the same transaction as the insert; repo invite sets its own expiry afterwards.
The on-open conversion of Julian config and user mtimes is now best-effort: while another connection holds the write lock, Open skips it and a later open converts the rows, instead of failing. (user: [email protected], size: 11776) [annotate] [blame] [check-ins using]
package checkout import ( "context" "fmt" "os" "path/filepath" "strconv" "strings" "github.com/danmestas/go-libfossil/internal/content" libfossil "github.com/danmestas/go-libfossil/internal/fsltype" "github.com/danmestas/go-libfossil/internal/hash" "github.com/danmestas/go-libfossil/internal/manifest" "github.com/danmestas/go-libfossil/internal/vfile" ) // extractSingleFile expands a blob and writes it to the checkout directory. // Skips writing if dryRun is true. func (c *Checkout) extractSingleFile( pathname string, blobRid int64, isexe int64, dryRun bool, ) error { if dryRun { return nil } data, err := content.Expand(c.repo.DB(), libfossil.FslID(blobRid)) if err != nil { return fmt.Errorf("checkout.Extract: expand blob for %s: %w", pathname, err) } fullPath, err := c.safePath(pathname) if err != nil { return fmt.Errorf("checkout.Extract: path traversal in %s: %w", pathname, err) } parentDir := filepath.Dir(fullPath) if err := c.env.Storage.MkdirAll(parentDir, 0o755); err != nil { return fmt.Errorf("checkout.Extract: mkdir %s: %w", parentDir, err) } perm := os.FileMode(0o644) if isexe != 0 { perm = 0o755 } if err := c.env.Storage.WriteFile(fullPath, data, perm); err != nil { return fmt.Errorf("checkout.Extract: write %s: %w", fullPath, err) } return nil } // Extract writes files from the specified checkin to disk via simio.Storage. // Populates vfile, updates vvar checkout/checkout-hash to rid. // // If opts.DryRun is true, skips writing files but still calls observer and callback. // If opts.Force is false (default), refuses before changing anything when the // current version has unsaved changes or an untracked file would be // overwritten (see checkSafeToExtract). // // Panics if c is nil (TigerStyle precondition). func (c *Checkout) Extract(rid libfossil.FslID, opts ExtractOpts) error { if c == nil { panic("checkout.Extract: nil *Checkout") } // Start observer ctx := c.obs.ExtractStarted(context.Background(), ExtractStart{ Operation: "extract", TargetRID: rid, }) var filesWritten int var extractErr error defer func() { c.obs.ExtractCompleted(ctx, ExtractEnd{ Operation: "extract", TargetRID: rid, FilesWritten: filesWritten, Err: extractErr, }) }() // Check before LoadVFile: it replaces the current version's rows, and a // refusal must leave the checkout exactly as it was. if !opts.Force && !opts.DryRun { if err := c.checkSafeToExtract(rid); err != nil { extractErr = err return extractErr } } // The files come from the target's manifest, so a dry run never has to // load them into vfile; only a real extract replaces the file list. vfRows, err := c.extractRows(rid) if err != nil { extractErr = err return extractErr } if !opts.DryRun { if err := c.replaceFileList(rid, opts.Force); err != nil { extractErr = err return extractErr } } for _, row := range vfRows { if err := c.extractRow(ctx, row, opts); err != nil { extractErr = err return extractErr } filesWritten++ } if opts.DryRun { return nil } if extractErr = c.finalizeExtract(rid, opts.Force); extractErr != nil { return extractErr } if opts.SetMTime { extractErr = c.setFileMTimes(rid) } return extractErr } // finalizeExtract looks up the blob UUID for rid and updates the vvar // checkout/checkout-hash entries. // It also ends a pending merge on a switch to another version or a forced // extract, but not on a plain re-extract of the version already checked out, // which fossil's checkout does not treat as a switch. func (c *Checkout) finalizeExtract(rid libfossil.FslID, force bool) error { prior, _, err := c.Version() if err != nil { return fmt.Errorf("checkout.Extract: %w", err) } endMerge := force || prior != rid var uuid string err = c.repo.DB().QueryRow("SELECT uuid FROM blob WHERE rid = ?", int64(rid)).Scan(&uuid) if err != nil { return fmt.Errorf("checkout.Extract: query blob uuid: %w", err) } if err := setVVar(c.db, "checkout", strconv.FormatInt(int64(rid), 10)); err != nil { return fmt.Errorf("checkout.Extract: %w", err) } if err := setVVar(c.db, "checkout-hash", uuid); err != nil { return fmt.Errorf("checkout.Extract: %w", err) } if !endMerge { return nil } // A switch ends any pending merge, as fossil's checkout does; a stale // vmerge row would give the next commit a bogus merge parent. if _, err := c.db.Exec("DELETE FROM vmerge"); err != nil { return fmt.Errorf("checkout.Extract: clear vmerge: %w", err) } return nil } // replaceFileList loads version rid into vfile. A forced extract first drops // every row, the current version's included, as fossil's checkout --force // does, so no pending add, rename or removal survives it; otherwise rows the // version already has are kept (LoadVFile). func (c *Checkout) replaceFileList(rid libfossil.FslID, force bool) error { if rid <= 0 { panic("checkout.replaceFileList: rid must be positive") } if force { if _, err := c.db.Exec("DELETE FROM vfile"); err != nil { return fmt.Errorf("checkout.Extract: clear vfile: %w", err) } } if err := c.LoadVFile(rid, true); err != nil { return fmt.Errorf("checkout.Extract: %w", err) } return nil } // extractRow is one vfile row Extract writes to disk. type extractRow struct { pathname string blobRid int64 isexe int64 } // extractRows lists version rid's files with the blob each one's content is // in, from its manifest. It refuses a version whose content is missing // (resolveFiles), before anything is written. func (c *Checkout) extractRows(rid libfossil.FslID) ([]extractRow, error) { if rid <= 0 { panic("checkout.extractRows: rid must be positive") } files, blobRIDs, err := c.resolveFiles(rid) if err != nil { return nil, fmt.Errorf("checkout.Extract: %w", err) } out := make([]extractRow, len(files)) for i, f := range files { isexe := int64(0) if strings.Contains(f.Perm, "x") { isexe = 1 } out[i] = extractRow{pathname: f.Name, blobRid: int64(blobRIDs[i]), isexe: isexe} } return out, nil } // extractRow writes one file and reports it to the observer and the // caller's callback. func (c *Checkout) extractRow( ctx context.Context, row extractRow, opts ExtractOpts, ) error { if err := c.extractSingleFile( row.pathname, row.blobRid, row.isexe, opts.DryRun, ); err != nil { return err } c.obs.ExtractFileCompleted(ctx, row.pathname, UpdateAdded) if opts.Callback != nil { if err := opts.Callback(row.pathname, UpdateAdded); err != nil { return fmt.Errorf("checkout.Extract: callback for %s: %w", row.pathname, err) } } return nil } // checkSafeToExtract refuses an Extract that would destroy work the user has // not committed. It follows fossil's checkout command, which refuses while the // current version has unsaved changes (edits, adds, removals, renames, // merges) and prompts before overwriting a file it does not track; a library // cannot prompt, so it refuses. It runs before anything in the checkout // changes. // // One relaxation: an edit whose content already equals what target would // write loses nothing, so it does not block. That matters because Create // records the tip as checked out before any file is written, so a directory // already holding an older version looks edited relative to the tip. func (c *Checkout) checkSafeToExtract(target libfossil.FslID) error { if target <= 0 { panic("checkout.checkSafeToExtract: target must be positive") } current, _, err := c.Version() if err != nil { return fmt.Errorf("checkout.Extract: %w", err) } files, err := manifest.ListFiles(c.repo, target) if err != nil { return fmt.Errorf("checkout.Extract: %w", err) } targetHash := make(map[string]string, len(files)) for _, f := range files { targetHash[f.Name] = f.UUID } if current > 0 { atRisk, err := c.firstChangeAtRisk(current, targetHash) if err != nil { return fmt.Errorf("checkout.Extract: %w", err) } if atRisk != "" { return fmt.Errorf( "checkout.Extract: file %s has local changes; use Force to overwrite", atRisk, ) } } return c.checkNoUntrackedClobber(current, files) } // firstChangeAtRisk scans version vid and returns the first changed file // whose change extracting target would lose, or "" when there is none. func (c *Checkout) firstChangeAtRisk( vid libfossil.FslID, targetHash map[string]string, ) (string, error) { if vid <= 0 { panic("checkout.firstChangeAtRisk: vid must be positive") } if targetHash == nil { panic("checkout.firstChangeAtRisk: nil targetHash") } if err := c.refreshChanged(vid); err != nil { return "", err } // A pending merge is unsaved work in its own right: the vmerge record // would be dropped even if every merged file already matches target. var merging bool if err := c.db.QueryRow( "SELECT EXISTS(SELECT 1 FROM vmerge)", ).Scan(&merging); err != nil { return "", fmt.Errorf("query vmerge: %w", err) } rows, err := vfile.Load(c.db, int64(vid)) if err != nil { return "", err } for _, r := range rows { change := vfile.Classify(r, false) if change == ChangeNone { continue } // Only a plain edit (no merge pending, not a merge state) of a file // target also writes can be safe; its content decides. if !merging && change == ChangeModified && !r.PendingMerge() && !r.ModeChanged() { if uuid, ok := targetHash[r.Pathname]; ok { clobber, err := c.wouldClobber(r.Pathname, uuid) if err != nil { return "", err } if !clobber { continue } } } return r.Pathname, nil } return "", nil } // wouldClobber reports whether writing the artifact uuid to pathname would // destroy content: true only when a file is there and differs from it. func (c *Checkout) wouldClobber(pathname, uuid string) (bool, error) { if !hash.IsValidHash(uuid) { panic("checkout.wouldClobber: invalid uuid for " + pathname) } fullPath, err := c.safePath(pathname) if err != nil { return false, nil // Extraction rejects the path itself, with a better error. } data, err := c.env.Storage.ReadFile(fullPath) if err != nil { if os.IsNotExist(err) { return false, nil } return false, fmt.Errorf("checkout.Extract: read %s: %w", fullPath, err) } return hash.ContentHash(data, uuid) != uuid, nil } // checkNoUntrackedClobber returns an error if a file that version current // does not track exists on disk where target would write different content. // current may be 0 (nothing checked out yet). func (c *Checkout) checkNoUntrackedClobber( current libfossil.FslID, files []manifest.FileEntry, ) error { if current < 0 { panic("checkout.checkNoUntrackedClobber: negative current") } tracked, err := c.trackedPaths(current) if err != nil { return err } for _, f := range files { if tracked[f.Name] { continue } clobber, err := c.wouldClobber(f.Name, f.UUID) if err != nil { return err } if clobber { return fmt.Errorf( "checkout.Extract: untracked file %s has local changes; use Force to overwrite", f.Name, ) } } return nil } // trackedPaths returns the set of pathnames vfile holds for version vid. func (c *Checkout) trackedPaths(vid libfossil.FslID) (map[string]bool, error) { rows, err := c.db.Query("SELECT pathname FROM vfile WHERE vid = ?", int64(vid)) if err != nil { return nil, fmt.Errorf("checkout.Extract: query vfile: %w", err) } defer rows.Close() tracked := make(map[string]bool) for rows.Next() { var name string if err := rows.Scan(&name); err != nil { return nil, fmt.Errorf("checkout.Extract: scan vfile: %w", err) } tracked[name] = true } if err := rows.Err(); err != nil { return nil, fmt.Errorf("checkout.Extract: iterate vfile: %w", err) } return tracked, nil }