0003 — quarry implementation plan

Companion to 0001-quarry-forge.md (the spec) and 0002-forgejo-salvage.md (the UI salvage map). This document owns what the spec deliberately does not: concrete technology choices, build order, testing strategy, and observability wiring. Decisions here are revisable without touching the spec; decisions that would change the spec's architecture go back through 0001.

0. Guiding constraints (inherited, not restated)

• 0001 Principles 1–7 govern every phase; the ops layer (0001 §9) is the seam everything routes through; the disposability test (0001 §11) is the merge gate from phase 0 onward.
• The smol contract: each package is a program that can be finished.
• 0002 §4 mechanics govern all Forgejo salvage (ledger first, cleanroom sanitizer, resolver inversion).

1. Build order

The organizing rule: the spec's riskiest claims get proven by a tracer bullet before any breadth work, and the three enforcement mechanisms (disposability harness, ops registry, provenance ledger) are built first, because retrofitting a merge gate onto an existing codebase never works.

Phase 0 — rails before features

• Ops registry core + OpenAPI generator + the import-graph CI assertion. Empty registry, one dummy op, spec generation working end to end.
• Disposability harness scaffold: fixture corpus of real converted stores (sqlite, tcl, plus a synthetic wiki/ticket/forum/chat-heavy fixture — the synthetic fixture is worth real effort; it is the corpus's coverage), ingest → snapshot → wipe → re-ingest → semantic-diff loop, in CI from day one even while it covers a single page.
• Provenance ledger + CI check (0002 §4 item 1).
• Go module skeleton with the boundaries drawn: internal/ops (sole importer of data planes), internal/ingest, internal/markup and sibling salvage packages, internal/web, internal/api.

Exit: CI runs all three gates; a dummy op renders a dummy page and appears in a generated OpenAPI document.

Phase 1 — the tracer bullet

One store → watcher (validated-prefix fold via the zit subprocess, zit#17 stat-and-reopen discipline from the start) → embedded NATS event → per-node SQLite fold → one op (timeline.list) → one HTML page and its generated API endpoint, footer with versions and render timing.

Proves the four load-bearing claims at minimum cost: log-as-event-stream, embedded-index-as-derived-state, registry-generates-spec, disposability-as-gate.

Phase 2 — read plane breadth

0002 §4's extraction order: markup first (cleanroom sanitizer + conformance corpus before anything renders) → base layout → fleet → repo home/files → timeline/check-in → diff stack → graph → wiki render → search (FTS5) → activity/stats. Each page lands against the per-page definition of done (0002 §4 item 5) and grows the disposability snapshot set incrementally.

Phase 3 — liveness

Datastar SSE wiring, chat transport + forum archival, notify integration. First real test of Principle 3's refresh-fallback discipline.

Phase 4 — sync fronting + leases

/xfer fronting, then the 0001 §3 lease machinery. Deliberately last in v1: it is the only distributed-systems-hard part, and everything before it works single-node.

v2/v3 then follow 0001 §10's staging. Each phase ends with the harness green and a docs/ page generated from the spec — documentation is a phase exit criterion, not a v-final activity.

2. Technology choices

Each entry: choice, rationale, escape hatch. These are ADR-grade decisions recorded compactly; split any into a standalone ADR when it accretes real debate.

Area	Choice	Rationale	Escape hatch
SQLite driver	ncruces/go-sqlite3	Upstream SQLite compiled to Wasm (wazero): real SQLite behavior, fast tracking of releases, no CGO (cross-compilation stays trivial), FTS5 enabled, and a Go-implementable VFS surface — the scale-out hook if node indexes ever want S3-backed hydration. Preferred over modernc (transpiler artifacts, historic concurrency edges) and mattn (CGO). Known costs: Wasm boundary tax on write-heavy paths (fits our read-dominated profile), amd64/arm64-only compiler (interpreter elsewhere), ~tens-of-ms startup compile, narrow bus factor (mitigated: unmodified upstream SQLite makes a forced move driver-swap-shaped, not data-shaped).	mattn (CGO) or modernc; schema is plain SQLite either way
HTTP router	stdlib http.ServeMux (1.22+ patterns)	The ops layer already does what chi would add; a router dependency is a shallow module here.	chi, mechanically
Ops registry	Hand-rolled on stdlib + generics (~1.5k lines total)	The registry is quarry's spine; you don't build the central abstraction on someone else's framework. The flat, form-expressible struct constraint (0001 §9) makes the reflector genuinely small. Pieces: generic Register[In,Out] over a descriptor map (~150 lines, explicit RegisterAll() wiring); JSON codec (stdlib); form/URL-values codec (~150 lines, needed for HTML regardless of framework); path-param binding onto ServeMux wildcards; OpenAPI reflector (~300–400 lines); tag-driven validator shared by both codecs; instrumentation wrapper around Run.	Huma (danielgtaylor/huma) — our Run signature is already its handler shape, so migration is mechanical
OpenAPI document types	Reuse dagnats' OpenAPI 3.1 synthesis types (currently internal/openapi in dagnats — promotion to a public package is a prerequisite, see §5)	One shared, tested document-assembly layer; quarry writes only the struct-reflection front half; both projects emit consistent 3.1 documents.	Hand-roll the back half too (~200 more lines)
Spec correctness gate	Off-the-shelf OpenAPI validator lints the generated spec in CI (dev-time tool, not a runtime dep); every registered op's fixtures round-trip through both codecs	Closes "our generator emits a subtly invalid spec" without a runtime framework.	—
Markup	goldmark + bluemonday + chroma	Per 0002 §2; sanitizer policy is cleanroom with the two-lineage conformance corpus + fuzz target.	—
Embedded NATS	nats-server as a library	The dagnats/notify pattern, already proven in this stack.	External NATS URL (already the scale-out flag)
litestream	Supervised sidecar	firestorm-proven shape; embedding buys nothing.	—
Parquet writing	Deferred with the interchange lake (0001: scale-out)	No Parquet dependency in v1 at all. When the lake lands, prefer writing via a query engine's COPY TO over a standalone parquet library — decide then.	—
Frontend	Datastar (only default-allowed framework, 0001 Principle 2) + html/template	Per 0001/0002.	Explicit per-case exception process
zit access	Subprocess boundary — the zit binary is the only store interface, reads and writes (0001 §2)	zit is Zig, quarry is Go; the versioned CLI is the whole contract, so releases decouple and quarry stays CGO-free. Most page data serves from quarry's indexes; on-demand reads (file-at-checkin, diff, annotate) pay ~1–5ms spawn, off the hot path and behind caching. Quarry's CI tests against pinned zit versions and each release declares a minimum zit version.	C-ABI read library exported from zit — revisit only with profiling evidence

3. dagnats integration

dagnats provides durable HTTP endpoints (HTTP trigger → event-sourced run → engine-resolved respond step) with OpenAPI 3.1 synthesis from trigger defs + declared schemas. Division of labor:

• Reads never route through dagnats. Two reasons that outrank latency: durable execution persists every request as run history (page-view-rate event-log garbage, retention becomes a problem), and 0001 §3 commits to "NATS unreachable → reads continue" — reads through dagnats would make a NATS outage a forge outage. Query ops are in-process function calls over the local index.
• Command ops that want durability get it inside Run. Chat archival, webhook dispatch, CI verdict minting — anything with retry/DLQ semantics — launches a dagnats workflow behind the seam. The registry stays the single OpenAPI source; no split-brain between two generators.
• CI/CD (0001 §6) uses dagnats natively, as spec'd.

4. Testability — one strategy per risk class

• Derived-state correctness: the disposability harness (0001 §11) is the keystone; every page and API response feeds it.
• Ops: query ops are pure functions over index state → golden tests are generated from the registry (enumerate ops, run fixtures, snapshot). New op = automatic coverage.
• The lease: deterministic simulation. Abstract the clock and the KV behind interfaces so the 0001 §3 expiry-during-write interleavings can be exhaustively driven in simulated schedules — no sleeps, no flakes. This is the one place in the system where a rare race matters.
• Sanitizer: the two-lineage conformance corpus + fuzz target (0002 §2).
• No-JS baseline: nearly free — pages are server-rendered, so the snapshot harness is the no-JS test. Plus one template lint: every data-on has a form/anchor sibling.
• Watcher: property tests on cursor loss / prefix re-validation (a lost cursor must converge to identical index state — a mini-disposability property).

5. Observability

Lean on dagnats: quarry imports dagnats' observe package for OTel exporter wiring, resource attributes, and conventions rather than building its own setup. Command ops propagate W3C trace context into the workflows they launch (dagnats' bridge already speaks tracecontext), so one stitched trace runs from HTTP request through the durable run.

Quarry adds only what dagnats cannot see:

• The ops-layer wrapper: span + latency histogram + counter per op call, labeled by op name (in-process reads never touch dagnats).
• Three gauges: ingest lag per store (cursor vs. log head — the one number that says quarry is healthy), lease state per store, and JetStream consumer lag.
• Prometheus /metrics; structured slog throughout; the Principle-1 footer render timing is the same measurement plumbed to metrics — one clock, two consumers.

6. Prerequisites and upstream work

• Fossil interop is a stated major priority — it is the adoption front door (Fossil users are the first expected adopters). Current state, verified in the zit tree 2026-08-14: wire-level interop already works — zit clones from a live Fossil server over the RFC'd sync protocol with hand-verified identical leaves, and the sync path handles all artifact types (wiki, ticket, forum, event, attachment). The open gap is consuming a bare .fossil file without a running server (zit's docs/plans/FOSSIL-IMPORT-PLAN.md has the route analysis; zit#1 is the adjacent issue; not yet built). Consequence for quarry: a Fossil user's first contact is zit clone of their existing served repo — no converter — and quarry can even front Fossil repos as continuously-synced mirrors. The .fossil-file reader is upstream zit work, prioritized accordingly; quarry's fixture tooling should use the serve-and-clone path meanwhile.
• zit: #59 (append-append flock), #17 (inode-swap reopen), #53 (clone batching) — already filed, per 0001 §10.
• zit, new (falls out of the subprocess boundary): a machine-consumable read-verb surface — at minimum "emit artifacts since cursor N with prefix validation" for the watcher, plus stable, parseable output for file-at-checkin / diff / annotate. The watcher verb's payload is raw artifacts in the RFC'd artifact format (0001 §2): zit resolves delta chains and streams whole artifacts; no new output schema is invented, and the container layout stays private. Audit what today's CLI already provides before filing; the watcher verb is the one most likely missing, and it is a phase-1 blocker.
• dagnats: promote internal/openapi to a public package so quarry can import the 3.1 document types (§2). Small PR, no behavior change.
• notify: wiring per its own RFC; no changes requested.

7. Risks

• ncruces bus factor — mitigated by unmodified-upstream-SQLite (see §2).
• Hand-rolled reflector correctness — mitigated by the CI validator lint
  • codec round-trip tests (§2).
• Synthetic fixture coverage — the disposability harness is only as good as its corpus; the wiki/ticket/forum/chat-heavy fixture needs deliberate investment in phase 0, not incidental accretion.
• Datastar no-JS discipline — enforced by the template lint and per-page definition of done, but the first genuinely live page (chat) will test whether refresh-fallback stays honest under feature pressure.