Artifact 5f75119881b9a3bb8f849549ef66b2d3bd348114dab408a181704a195f7ce322:
- File
db/scan.go
— part of check-in
[2c2e5e74b6]
at
2026-10-09 15:35:38
on branch trunk
— Store check-in, link and tag rows as fossil does (#258) (#261)
* fix(manifest): store check-in, link and tag rows as fossil does (#258)
A rebuilt or locally written repository differed from fossil's in three derived values: event and plink times in their last digits, event.omtime left NULL, and a valueless tag stored as '' instead of NULL.
Fossil stores a tag time as SQLite's julianday() bound as a double, but writes event, plink, attachment and forumpost times into SQL text with its own printf("%.17g"), which emits 16 digits in long double arithmetic. TimeToJulian is now julianday() exactly (whole milliseconds over 86400000) and TimeToTextJulian ports fossil's printf conversion, so both match fossil bit for bit. The conversion follows builds where long double is a double (macOS, Windows); fossil itself differs across platforms in the last digit.
Check-in events record omtime, tag writes bind NULL for an empty value, the readers of event and plink times under the ncruces driver rebuild the text form, and the CLI's tag listing reads NULL values.
* test(db): sweep both stored julian forms; scan the text form from driver times (#258)
* test(verify): compare text times bit for bit only where fossil computes them in double (#258) (user: [email protected], size: 6166) [annotate] [blame] [check-ins using]
package db import ( "time" "github.com/danmestas/go-libfossil/internal/fsltype" ) // julianEpoch is the Julian Day Number for the Unix epoch (1970-01-01 12:00:00 UTC). const julianEpoch = 2440587.5 // millisPerDay is the number of milliseconds in one day. Every mtime this // codebase writes is at millisecond resolution (TimeToJulian rounds to whole // millis), so it is the grid the time.Time branch recovers a scanned instant // onto. const millisPerDay = 86400.0 * 1000.0 // ScanJulianDay converts a scanned mtime value to a float64 Julian Day Number. // SQLite drivers return mtime differently: modernc returns float64, ncruces // returns time.Time for DATETIME/TIMESTAMP/DATE columns. Scan the column into // an `any` variable, then pass it here. // // The float64 branch returns the scanned value verbatim, and that exactness is // load-bearing rather than incidental. A float64 read off the row IS the stored // value, so any "canonicalizing" transform can only move it away from what the // row holds. Two consumers compare a scanned mtime against other stored mtimes // and would silently return a wrong answer (never an error) if the value were // perturbed even by one ulp: // // - Timeline's pagination cursor, whose predicate `e.mtime < ? OR (e.mtime = ? // AND b.rid < ?)` needs the cursor to compare exactly equal to the row it // came from. Raise it by an ulp and that row repeats as the next page's // first row; lower it and rows are skipped. // - tag propagation's `tagxref.mtime < ?`, where SQLite compares the bound // value against raw stored floats. // // Neither is hypothetical for values this library did not write: a value // another writer computed differently, or at finer than millisecond // resolution, is not on the grid TimeToJulian writes, and snapping it to that // grid changes it. // // The time.Time branch is the one case where a transform is required rather than // harmful — see timeToJulian. func ScanJulianDay(v any) (float64, bool) { switch v := v.(type) { case float64: return v, true case time.Time: return timeToJulian(v), true case int64: // Integer columns only appear as the COALESCE(mtime, 0) "no mtime" // sentinel; pass it through so callers' `mtime != 0` guard holds. return float64(v), true default: return 0, false } } // ScanTextJulianDay is ScanJulianDay for a column fossil writes as SQL text: // event, plink, attachment and forumpost times. Those hold the 16-digit form // (fsltype.TextJulianDayFromMillis), so a time.Time a driver hands back is // converted to that form rather than to the exact julianday() ScanJulianDay // gives for tagxref.mtime. A float64 passes through unchanged either way. func ScanTextJulianDay(v any) (float64, bool) { if t, ok := v.(time.Time); ok { return fsltype.TextJulianDayFromMillis(t.Round(time.Millisecond).UTC().UnixMilli()), true } return ScanJulianDay(v) } // ScanTime converts a scanned mtime value to time.Time. // Same driver-compatibility rationale as ScanJulianDay. func ScanTime(v any) (time.Time, bool) { switch v := v.(type) { case time.Time: return v, true case float64: return julianToTime(v), true case int64: return julianToTime(float64(v)), true default: return time.Time{}, false } } // ScanInt converts a scanned value to int. // SQLite drivers differ on BOOLEAN columns: modernc returns int64, // ncruces returns bool. Scan the column into an `any` variable, then pass it here. func ScanInt(v any) (int, bool) { switch v := v.(type) { case int64: return int(v), true case int: return v, true case bool: if v { return 1, true } return 0, true case float64: return int(v), true default: return 0, false } } // julianToTime converts a julian day to a time.Time. Note this truncates toward // zero where timeToJulian rounds; the two are not inverses at sub-millisecond // resolution. Pre-existing and left alone deliberately — ScanTime's consumers // are display paths with no ordering boundary, and changing a conversion used // there is a separate change from the exactness ScanJulianDay's callers need. func julianToTime(jd float64) time.Time { millis := int64((jd - julianEpoch) * millisPerDay) return time.UnixMilli(millis).UTC() } // julianDayFromMillis converts whole milliseconds since the Unix epoch to a // Julian Day Number with the same function TimeToJulian writes with, so a // value recovered through it reproduces the original write bit-for-bit. func julianDayFromMillis(millis int64) float64 { return fsltype.JulianDayFromMillis(millis) } // timeToJulian converts a driver-scanned time.Time back to a julian day // float64. Rounds to the nearest millisecond rather than truncating: the // ncruces/WASM sqlite driver hands back DATETIME columns as a time.Time // via its own internal REAL conversion, which carries sub-millisecond // noise (observed ~13us) around the true millisecond-aligned instant every // mtime in this codebase is written as. A plain UnixMilli() truncates that // noise into a full missed millisecond when it lands just below the true // value; rounding recovers the exact intended millisecond, which is what // makes ScanJulianDay(scannedTime) reproduce the same float64 bit-for-bit // as the value TimeToJulian originally wrote, across both drivers. This // matters beyond cosmetics: Timeline's pagination cursor is built from // exactly this value and depends on it matching the stored row exactly. // // The recovery is exact only for mtimes written at millisecond resolution, // which is every mtime this codebase writes. An mtime written by upstream // Fossil via julianday() can carry finer precision, and this branch cannot // reproduce it — event.mtime is declared DATETIME, so ncruces has already // converted it through a time.Time before this code sees the value. That is a // driver limitation with no fix available here; rounding the float64 branch to // match would not recover the precision, only discard it on both drivers and // break the exact-cursor invariant in the process. func timeToJulian(t time.Time) float64 { return julianDayFromMillis(t.Round(time.Millisecond).UTC().UnixMilli()) }