go-libfossil: Artifact [3a46828b34]

Artifact 3a46828b340acd6e22924085da8736bb60d7144de843d6ba0852d5e1f5be09f7:

File docs/site/content/docs/architecture.md — part of check-in [6273a0750d] at 2026-07-22 23:03:24 on branch trunk — fix: correct go.mod module path to go-libfossil (#76) (#117)
* fix: correct go.mod module path to go-libfossil (#76)
The module directive still declared github.com/danmestas/libfossil after the repository was renamed to go-libfossil. pkg.go.dev/godoc scraping matches the declared module path against its hosting path, so the stale path broke SDK doc generation.
Rename the module path across all four in-repo modules (root, the two db/driver submodules, observer/otel, and docs/site) and every internal import, plus the release workflow, the doc-generation script, and the docs.
No tag has been published under the go-libfossil path yet, so the cross-module requires cannot resolve from the proxy. Add replace directives pointing the in-repo requires at their local directories so GOWORK=off builds and the CI matrix resolve without a published version. These replaces are ignored by downstream consumers and can be dropped once new-path tags are cut.
* Guard releases against replace directives; regenerate SDK docs
The go-libfossil rename bridges the in-repo modules with `replace` directives. Go refuses a main module that carries them, so a tag cut while they are present breaks `go install .../cmd/libfossil@<tag>` and every `@version` fetch of the driver submodules. Add a Release workflow step that fails the job before anything is published if any tracked go.mod still contains a replace directive, so the bridge is a CI-enforced TODO rather than a silent landmine. The step comment records what unblocks a release: re-tag the submodules with the corrected module paths, bump the root requires, then drop the replace blocks.
Regenerate the SDK reference docs. The previous pass only substituted URLs over the pre-existing generated files, leaving stale signatures (Repo.Timeline was still documented as taking LogOpts rather than TimelineOpts). Note that gomarkdoc must be run outside a git worktree — inside one it silently strips every source link.
Also reword the README rename note, which read as though the import path had not changed, and re-sort the ncruces require block. (user: [email protected], size: 11446) [annotate] [blame] [check-ins using]

title: Architecture

weight: 20

Overview

libfossil is a pure-Go library and CLI that reads and writes Fossil SCM's .fossil SQLite repositories. The public surface (package libfossil) exposes Repo, Create, Open, Clone, checkout, and check-in operations; internals live under internal/. SQLite access is mediated by a pluggable driver layer in db/, with two shippable drivers (modernc, ncruces) selected by blank import. Instrumentation is pulled in by implementing the SyncObserver / CheckoutObserver interfaces; an optional OTel adapter lives in its own submodule. A deterministic simulator under dst/ plus a BUGGIFY fault-injection harness in simio/ exercise concurrent repo operations under a seed.

Module layout

This is a multi-module workspace (go.work) with four go.mod files: the root, two driver modules, and the OTel observer. Layout:

Root package libfossil — thin facade over internal/repo, internal/sync, and internal/checkout. Files: fossil.go (Create, Open, Clone), repo.go (Repo handle + DB accessor), checkout.go, repo_checkout.go, repo_merge.go, repo_sync.go, repo_history.go, repo_admin.go, repo_extras.go, transport.go, observer.go, types.go, errors.go, julian.go.
cli/ — Kong command definitions. repo.go aggregates 35+ subcommands (new, clone, ci, co, timeline, diff, merge, tag, branch, uv, stash, bisect, annotate, user, etc.). Intended to be embedded by any main that wants a fossil-shaped CLI.
cmd/libfossil/ — the shipped binary (main.go, 25 lines). Wires Kong, blank-imports db/driver/modernc, and dispatches to cli.
internal/ — private implementation, grouped by concern:
- Storage layer: blob, content, manifest, deck, delta, hash, fsltype, repo (schema + DB boot), tag.
- Working-tree operations: checkout (checkin, extract, update, revert, fork, rename, status, vfile), merge (three-way, ancestor, detect, resolve, strategies), diff, annotate, undo, stash, bisect, branch, path.
- Sync / transport: sync (client, handler, clone, session, ckin-lock, serve-http, table-sync, UV), xfer (card codec).
- Housekeeping: auth, shun, verify, search, uv, testdriver.
db/ — SQLite abstraction (see below).
db/driver/modernc, db/driver/ncruces — separate modules, each registers exactly one database/sql driver.
observer/otel/ — separate module implementing the observer interfaces against OpenTelemetry.
dst/ — deterministic simulator (simulator.go, node.go, network.go, peer_network.go, mock_fossil.go, invariants.go, event.go) with scenario-style tests.
simio/ — deterministic IO primitives: SimClock, seeded Rand, in-memory storage, and the global Buggify switch.
testutil/ — shared test helpers for the rest of the tree.

Storage model

A .fossil file is a SQLite database. All repository state — blobs, manifests, tags, branches, unversioned files, tickets, wiki, config, users — lives in that file. libfossil does not maintain any sidecar state for the repo itself (a working tree has its own SQLite DB).

Content is stored content-addressed in the blob table, keyed by either SHA1 or SHA3-256 UUIDs (hash mode is per-project). Every blob is zlib-compressed with a 4-byte big-endian uncompressed-size prefix; internal/blob.Compress and Decompress handle that framing. Blobs may be stored as full content or as deltas against a parent; reconstruction walks the chain and applies deltas via internal/content.Expand, with cycle detection. internal/content.Verify rehashes expanded content against the stored UUID.

Manifests are themselves blobs whose text describes a commit, check- out state, tag change, branch operation, wiki edit, ticket change, or cluster. internal/manifest and internal/deck parse and assemble manifests; "crosslinking" (internal/manifest/crosslink.go) is the step that projects a manifest's effects into derived tables (mlink, plink, event, tagxref, etc.) so queries like timeline and checkout work without re-parsing blobs.

Driver layer

db/ is a thin wrapper over database/sql. See db/doc.go. A driver registers at init time via db.Register(DriverConfig{Name, BuildDSN}); exactly one registration is allowed (a second call panics). db.Open / db.OpenWith look up the registered driver, run BuildDSN(path, pragmas) to produce the DSN, and then call sql.Open with that driver name. Both shipped drivers construct file:<path>?_pragma=k(v)&... DSNs — differing only in the database/sql driver name they register under ("sqlite" for modernc, "sqlite3" for ncruces).

Shipped drivers:

db/driver/modernc — default. Blank-imports modernc.org/sqlite, a pure-Go, CGo-free translation of SQLite with full filesystem access. This is what cmd/libfossil/main.go imports.
db/driver/ncruces — blank-imports github.com/ncruces/go-sqlite3, which runs SQLite as WebAssembly. Has a ncruces_js.go variant for GOOS=js / browser targets. Used for GOOS=wasip1 builds and for the test_ncruces build tag in the Makefile's driver matrix.

Sync protocol

libfossil implements Fossil's sync-over-HTTP protocol as a client / server pair in internal/sync. Wire framing (xfer "cards") lives in internal/xfer. Public entry points are re-exported from libfossil as Clone (fossil.go) and from repo_sync.go.

Client (internal/sync/client.go, clone.go, session.go): drives rounds until igot/gimme reconcile. Supports clone, pull, push, sync (both directions), plus unversioned-content (client_uv*.go) and config table (client_tablesync.go) sync.
Server (internal/sync/handler.go, serve_http.go, handler_uv.go, handler_tablesync.go): stateless per round; XferHandler plugs into http.ServeMux.
Check-in lock (ckin_lock.go): a ci-lock-<parent> row in the config table serialises concurrent check-ins against the same parent. Stale entries expire after DefaultCkinLockTimeout (60s).
Shun propagation (xdelete.go, internal/shun): deleted blobs are tombstoned and the tombstone propagates during sync.

Observer model

The core keeps instrumentation as a non-goal; hooks are plain Go interfaces on the public API. SyncObserver and CheckoutObserver in observer.go define lifecycle events (session start/end, per-round stats, extract/scan/commit phases). NopSyncObserver / NopCheckoutObserver are zero-cost defaults; StdoutSyncObserver / StdoutCheckoutObserver log to stderr for quick debugging.

For OpenTelemetry, observer/otel is a separate Go module so the core doesn't pull in the OTel SDK. Users import it explicitly when they want traces, metrics, or structured logs.

See Extension Points for the full observer contract and a worked example of a custom observer.

DST / simulation

dst/ runs many Fossil nodes deterministically under a seed. A Simulator (dst/simulator.go) owns a priority-queue event loop, a SimNetwork, a simio.SimClock, and a set of Node actors. Scenarios schedule clones, check-ins, merges, UV pushes, and partitions, and tests assert invariants (dst/invariants.go, crosslink_completeness_test.go).

Fault injection uses simio.Buggify(p) — a global probability gate that returns false in production but flips true with probability p during simulation. Sites like clone.emitCloneBatch.truncate in the sync code wrap calls behind Buggify so the simulator can exercise failure paths. SeededBuggify in dst adapts this to the BuggifyChecker interface used by the sync client.

See testing.md for running the DST matrix, seed sweeps, and the driver test matrix.