- Feature Name: `ci` (verdicts on landings: dagnats runs, a spec people can write, logs that outlive the runner, and doors for runners and CI quarry does not own) - Start Date: 2026-08-28 - Status: In progress — rungs 1–5 shipped, 7 (logs) and 8 (premerge, carry-forward, auto-accept) partly; 6 (runners page) and 9 open - Depends on: RFC 0001 §6 (dagnats is the engine — kept, this is its surface); RFC 0005 (event vocabulary — triggers and webhooks); RFC 0007 (protected branches — required checks configure beside its rules); RFC 0008 (roles — every new surface gates on capability letters) - Upstream: dagnats issues [#624](https://github.com/danmestas/dagnats/issues/624)–[#633](https://github.com/danmestas/dagnats/issues/633); each rung below names what it waits on, and most wait on nothing ## Summary [summary]: #summary Quarry's CI skeleton is live — check-ins trigger dagnats pipelines, a poller mirrors status, terminal runs can write `ci-pass`/`ci-fail` tag artifacts — but the accept button cannot see any of it, the pipeline format is raw engine JSON, there are no logs, and runners attach by folklore. This RFC turns the skeleton into a surface: a checks panel on every landing that the accept path reads; a small spec (`ci/quarry.yml`) compiled by the engine's own CI module; premerge testing that adds nothing to the DAG; logs captured by the engine, offloaded to S3, and read through one viewer; a Runners page whose tokens are how machines — including hosted provider fleets — attach; and two doors for CI quarry does not run: an external checks op in, and webhooks out. The stance sentence, already written in quarry's own source, stays the architecture: *dagnats is the engine, quarry the trigger and display* — and, this RFC adds, the policy. ## Motivation [motivation]: #motivation An accept today is a human reading a diff plus a clean-merge probe. Every other input to that judgment — does it build, do the tests pass, does the merged result still pass — either does not exist or exists where the accept cannot see it (`/r/{repo}/ci` lists runs; the landing page does not). Landings and strata are structurally complete and evidence-free. Meanwhile the parts that do exist speak engine-ese: a pipeline is a raw dagnats `WorkflowDef` in `ci/*.json`, which is a users' manual in the same way the old capability-letter table was (RFC 0008 §motivation). And a run that fails answers "failed" with nothing behind the word — no log, no failing step, no way to reproduce. ## Architecture: three layers [architecture]: #architecture-three-layers Each piece lives where somebody else would also use it: | Layer | Owns | Why there | |---|---|---| | **dagnats core** | run engine, retries, approval gates; the facilities: log lane (#624), lifecycle events (#625), per-step timestamps (#626), scoped/JIT worker tokens (#627), worker directory (#628), run labels and filters (#629), re-register hash (#630), queue-depth signal (#632) | every dagnats front-end needs them; logs and timestamps can only be captured inside the engine and worker SDK | | **dagnats CI module** (mounted in `dagnats serve`, #633) | `POST /v1/ci/compile` and `/v1/ci/validate` (the ci.yml compiler, errors as data); the worker set (dagger caller, Dagster adapter, later others); the annotation convention (#630) | forge-agnostic CI vocabulary — the module's GitHub front-end uses the identical compiler and workers; none of it knows what a landing is | | **quarry `internal/ci`** | triggers from quarry events; tip-pair premerge orchestration; the checks panel and accept gating; `check.report` and `ci.trigger` ops; the Runners page; verdict-tag policy | quarry semantics — landings, protected branches, the ledger's rules | Quarry speaks one URL. The compiler is never vendored and never shelled: compilation happens at the engine's own version behind the same base URL (`--dagnats`), so spec/engine skew is impossible by construction. The one quarry-shaped piece that runs on workers — the premerge checkout step — ships from the quarry repo, because its semantics (tip pairs, read-only credentials, verdict keying) are the forge's. ## The spec [spec]: #the-spec A repository opts into CI by committing a file. Two formats, one truth: - `ci/*.json` — raw dagnats `WorkflowDef`, honored exactly as today. The engine format is the truth; everything else compiles to it. - `ci/*.yml` — the small spec, compiled by the CI module at trigger time: ```yaml checks: test: { call: "test" } lint: { call: "lint" } build: { call: "build", needs: [test, lint], timeout: "20m" } ``` Rules, stated once and enforced by refusing to grow: 1. **No logic in the spec.** No expressions, no conditionals, no templating. A step names a task, its needs, and its limits; logic lives in the tasks, which is what a DAG engine is for. This is the trap that ate the incumbent's format, avoided by construction. 2. **Validation is a surface, not a failure mode.** The settings page and the empty CI page call `/v1/ci/validate` and render errors as data — line, field, message — before anything runs. A repository with a spec that does not compile shows the error where the runs would be, not a silent absence. 3. The empty CI page offers a starter file, the way the ticket editor offers its vocabulary. ## Triggers [triggers]: #triggers Four ways a run starts; the first exists today. 1. **Check-ins** — the watermark walk (`internal/ci/trigger.go`), unchanged in shape: new check-in, read the pipeline files at that check-in, compile if yml, start. Run input carries repo, check-in, branch, user, comment, clone URL — and, new, **labels** (`repo=`, `checkin=`, `branch=`) once dagnats #629 lands, so runs are findable and cancellable by what they are about. 2. **Landings** — opening or updating a landing triggers the target branch's pipelines against the landing branch (tip mode) or the merged tree (premerge mode, [below](#premerge)). A landing run is labeled `landing=`; a new push to the branch cancels the stale runs by label (#629) and starts fresh ones. 3. **By hand** — a `ci.trigger` op: run a named pipeline now, with inputs. Deploys, nightlies-on-demand, re-runs. Gated by `i` (RFC 0004: the check-in letter — if you may write code here, you may run its pipelines); the button on a red check row says "Re-run". 4. **From outside** — dagnats' own cron and external trigger envelopes cover schedules and foreign events; and the loop quarry already half-owns closes the fully-external case: outbound webhooks (`landing.opened`, check-in events, RFC 0005) are what a foreign CI subscribes to, tests, and answers via [`check.report`](#external-ci). ## Verdicts on landings [verdicts]: #verdicts-on-landings The Gerrit shape: the machine's opinion is a named thing beside the human's decision, never mixed into it. - **The checks panel.** Every landing shows one row per pipeline relevant to it: status chip, pipeline name, started/duration, a link into the run (quarry's log viewer once it exists; the dagnats console meanwhile). External checks ([below](#external-ci)) render in the same panel, their target URL standing in for the log link. Live via the SSE morphing the CI page already uses; the no-script baseline is load-time state plus refresh (RFC 0001 Principle 3). - **Required checks** configure per branch, beside RFC 0007's protection rules and gated the same way (`a`): a list of check names that must be green before an accept proceeds. The vocabulary is names, not runs — a required check with no run yet reads *waiting*, and blocks. - **The accept button says why.** Green: "Accept — checks green". Red or waiting: the control is disabled with the reason in words, exactly as protected branches already refuse — *"test is failing"*, *"build has not run"*. A control that would fail is not offered; one that names why is better than none (RFC 0006 §strata-UI, the rule landings already live by). - **Override is the owner's, said out loud.** A branch owner (RFC 0007) or an `a`-holder may accept over a red or missing check only with a stated reason, which is recorded as a landing comment before the accept is written. The record shows both the red check and the sentence that overrode it. - **Auto-accept when green** is a per-landing opt-in by someone who could accept it now were it green: the forge accepts on their behalf when the required set turns green, attributing the accept to them with an "auto-accepted: checks turned green" note. The freshness rule below applies — a moved target re-arms the wait rather than accepting stale evidence. This is the seed that grows into a merge queue; the queue data structure itself is deliberately deferred. ## Premerge: nothing enters the DAG [premerge]: #premerge-nothing-enters-the-dag The ledger is append-only, so the git-world trick — materialize a speculative merge ref, test it, garbage-collect it — is exactly wrong here. Premerge is a checkout: 1. A worker with **read-only credentials** clones (or reuses a cached clone), opens the target branch, and runs `zit merge `. The merged tree exists only in the worktree; a conflict is itself the verdict, with the tree as evidence. 2. The pipeline runs against that tree. **Nothing is ever committed.** The worktree is discarded. Ledger safety is structural, not policy: nothing was committed, and the credentials could not have pushed it anyway. A malicious pipeline can trash its own worktree and nothing else. **Verdicts attach to the pair of tips** `(branch, target)`, never to a check-in hash — the merge check-in does not exist yet, and the one accept eventually writes will hash differently. The consequences fall out: - **Freshness**: the target moving makes the pair stale; stale verdicts expire and the runs re-trigger (cancel-by-label, #629). Merge-queue semantics without a queue structure. - **Carry-forward**: after accept writes the real merge from the same pair, the verdict may be tagged onto it where [tags are opted in](#verdict-tags) — the portable record says this merge was tested before it existed. - **Strata**: landing a stack premerges rungs sequentially in one worktree; the whole-stack verdict falls out of the same mechanics. Tip mode is the default; premerge is opt-in per protected branch (`mode: tip | premerge` beside the RFC 0007 rule). The later optimisation — a pure zit verb taking two tips and answering with a merged tree tarball or a conflict report, so workers need no checkout at all — is an upstream request for when the checkout path shows its cost, not a blocker. Rejected: speculative store copies (a real check-in hash, but a useless one, at full-store cost) and test-after-accept with auto-revert (reverts are ledger pollution, and the point is that trunk never goes red). ## Logs [logs]: #logs Split at the engine's edge (#624, as rescoped there): dagnats owns the hot lane and the contract; retention belongs to the consumer — a forge that needs a verdict to stay explainable for years stores the logs next to the verdict. - **Hot lane (dagnats)**: the worker SDK's `LogOut`/`LogErr` writers stream step output as bounded chunks (`{seq, ts, stream: out|err|marker, data}`) on the `BUILD_LOGS` JetStream stream, `logs.{runID}.{stepID}`; a failure emits a `marker: failed` chunk so first-failure is recorded, not inferred. `GET /runs/{id}/logs?step=&after_seq=&follow=1&from=failure` reads and tails it (SSE on follow). Short TTL; the stream is a documented consumer contract beside #625, and the terminal `run.*` event is the drain trigger. - **Retention (quarry)**: on a run's terminal event — or the poller noticing, which is the same fact later — quarry drains the run's logs and stores them beside its own `ci_run` record, one object per run in the derived-data directory, with an S3-compatible target as configuration for fleets that want the objects elsewhere. Cold reads come back through the same viewer; it never knows which side answered. Per-step durations come from #626 (shipped: the engine persists `StartedAt`/`CompletedAt`). Quarry's viewer is server-rendered chunks plus an SSE tail, with the spine that makes logs bearable: collapsible per-step groups (`
`), the failed step first, jump-to-first-failure, a permalink to any line — and on every failure a printed **repro line** (`zit clone … && …`: the failing task, locally, no forge in the loop), which is this architecture's answer to SSH-into-the-build-VM. Failures that carry the annotation convention (#630) additionally pin onto the landing diff as file/line notes. ## Runners [runners]: #runners A **Runners** section at both scopes — `/fleet/settings` and repository settings — gated by RFC 0008's letters (`s` fleet, `a` repo): - **Mint a token** (#627), scoped to task types or queues, labeled with who and where. The page shows the one-liner to start a worker with it. Revoke from the same list. - **See what is attached** (#628): worker id, task types, hostname, transport, liveness. Hosted provider fleets — the Blacksmith class — attach through the same door, which is the point: a provider integrates once with dagnats and serves every dagnats front-end. Their two extras are upstream: single-use JIT registrations (#627) so a fresh VM per job never holds a reusable credential, and the queue-depth signal (#632) to size fleets without polling. A provider-run job is indistinguishable from a basement machine: same logs, same steps, same verdicts. ## External CI [external-ci]: #external-ci For CI quarry does not run — a Jenkins, a bespoke script, a mirror's pipeline — one op closes the loop: - **`check.report`**: check-in hash *or* tip pair, a context name, a state (`pending` / `success` / `failure` / `error`), a target URL, an optional summary. External checks render in the checks panel (target URL as their log link), count toward required checks when an administrator names them, and earn verdict tags only where tags are opted in. - Credentials are **reporter tokens**: minted and revoked on the Runners page, scoped to `check.report` alone. This is not a second permission system (RFC 0008 retired that): a token *authenticates*, and authorization still asks the effective capability set — a reporter token's set is simply fixed and one op wide. A compromised one can lie about a check (visible, attributable, revocable) and do nothing else. Subscribe (webhooks out) → test (their infrastructure) → report (`check.report`) is the entire bring-your-own-CI contract. ## Verdict tags [verdict-tags]: #verdict-tags Pass/fail as tag artifacts is quarry's own invention — CI history that clones with the repository — and it is also a write into the DAG on every run. So it is **opt-in per repository**: a Settings toggle, with the fleet default page saying what new repositories start as. Preserve mechanics: - Tags already written are ordinary immutable artifacts. Toggling off stops new writes and never shuns old ones. - Toggling on resumes from the next terminal run; there is no backfill. - The forge's `ci_run` table keeps full history either way; the tags are the portable subset. Premerge verdicts reach the DAG only via carry-forward at accept, since their subject does not exist before it. The `--ci-verdicts` flag becomes the fleet default this setting reads. ## Capability mapping [caps]: #capability-mapping Per RFC 0008, every surface is letters, stated here once: | Surface | Gate | |---|---| | See runs, checks, logs | the repository read gate, as `ci.list` today — CI is not more secret than the code | | `ci.trigger`, re-run | `i` | | Required-checks config, premerge mode | `a`, beside the RFC 0007 rules | | Runners page, tokens (repo / fleet) | `a` / `s` | | `check.report` | a reporter token, scoped to the op | | Accept over red, auto-accept arming | whoever could accept the landing (RFC 0007 owners; the override additionally requires the stated reason) | ## Build order [build]: #build-order | Rung | Work | Waits on | |---|---|---| | 1 | This RFC | — | | 2 | Checks panel on landings; required checks beside RFC 0007; disabled-accept wording; override-with-reason. Runs associate with a landing by its branch-tip check-in until rung 5's labels | nothing (status via today's poller) | | 3 | `check.report` + reporter tokens; `ci.trigger`; the external loop documented | nothing | | 4 | Spec: yml detection in the trigger, compile/validate via the CI module; starter file; errors-as-data on the settings and CI pages | #633 (#631 folded in) | | 5 | Landing triggers; labels on every run; cancel-superseded; events replace the poll | #625, #629 | | 6 | Runners page, both scopes | #627, #628 | | 7 | Log viewer (tail, groups, first-failure, permalinks, repro line, diff annotations) | #624, #626, #630 | | 8 | Premerge: the checkout step helper, per-branch mode, tip-pair verdicts, carry-forward, auto-accept-when-green | nothing upstream | | 9 | Provider polish: JIT tokens surfaced, queue-depth on the fleet page | #627 ext, #632 | Rungs 2 and 3 ship on today's engine with no upstream movement; that is deliberate — the accept gets its evidence first, everything else makes the evidence better. ## What this is not [not]: #what-this-is-not - **Not GitHub Actions compatible — yet, and never at the protocol level.** Workflow-file compatibility (an `act`-executing worker mapping quarry events to `push`/`pull_request`) is a plausible later module in the CI layer, explicitly **deferred** from this RFC: the native spec ships first, and any shim will be labeled a migration bridge with its fidelity limits stated. The GHA *runner protocol* — impersonating GitHub's Actions service so its runner binary or GHA-only farms attach — is out forever: enormous, undocumented, churning, and even the forges famous for Actions compatibility built their own runner instead. Providers get our documented door, which is smaller than the one they already passed through. - **Not a merge queue** — yet. Tip-pair freshness plus auto-accept-when- green covers the single-landing case; a batching, bisecting queue is a later RFC if contention ever demands it. - **Not an artifact or cache service.** Build artifacts and caches are the workers' business; a run's durable outputs are its logs and its verdict. - **Not post-accept enforcement.** Nothing here reverts, force-moves, or edits history on a red check after the fact; the gate is before the merge, and override is a human with a reason. ## Testing [testing]: #testing The harness grows a fake engine (an httptest server speaking `/workflows`, `/runs`, `/v1/ci/*`) so every rung is testable without NATS: required checks block and unblock the accept control; the override records its comment before the accept; a moved target expires a pair verdict; the premerge step refuses to run with pushable credentials; `check.report` rows render and count; tag opt-in writes on terminal runs and toggling off writes nothing. The premerge checkout step never invokes commit or push by construction, and its integration test against a real store asserts the ledger's artifact count is byte-identical before and after a full premerge run — including a run whose pipeline tries to commit.