- Feature Name: `ci` (verdicts on landings: dagnats runs, a spec people can
write, logs that outlive the runner, and doors for runners and CI quarry
does not own)
- Start Date: 2026-08-28
- Status: In progress — rungs 1–5 shipped, 7 (logs) and 8 (premerge, carry-forward, auto-accept) partly; 6 (runners page) and 9 open
- Depends on: RFC 0001 §6 (dagnats is the engine — kept, this is its
surface); RFC 0005 (event vocabulary — triggers and webhooks); RFC 0007
(protected branches — required checks configure beside its rules); RFC
0008 (roles — every new surface gates on capability letters)
- Upstream: dagnats issues [#624](https://github.com/danmestas/dagnats/issues/624)–[#633](https://github.com/danmestas/dagnats/issues/633);
each rung below names what it waits on, and most wait on nothing
## Summary
[summary]: #summary
Quarry's CI skeleton is live — check-ins trigger dagnats pipelines, a poller
mirrors status, terminal runs can write `ci-pass`/`ci-fail` tag artifacts —
but the accept button cannot see any of it, the pipeline format is raw
engine JSON, there are no logs, and runners attach by folklore. This RFC
turns the skeleton into a surface: a checks panel on every landing that the
accept path reads; a small spec (`ci/quarry.yml`) compiled by the engine's
own CI module; premerge testing that adds nothing to the DAG; logs captured
by the engine, offloaded to S3, and read through one viewer; a Runners page
whose tokens are how machines — including hosted provider fleets — attach;
and two doors for CI quarry does not run: an external checks op in, and
webhooks out.
The stance sentence, already written in quarry's own source, stays the
architecture: *dagnats is the engine, quarry the trigger and display* — and,
this RFC adds, the policy.
## Motivation
[motivation]: #motivation
An accept today is a human reading a diff plus a clean-merge probe. Every
other input to that judgment — does it build, do the tests pass, does the
merged result still pass — either does not exist or exists where the accept
cannot see it (`/r/{repo}/ci` lists runs; the landing page does not).
Landings and strata are structurally complete and evidence-free.
Meanwhile the parts that do exist speak engine-ese: a pipeline is a raw
dagnats `WorkflowDef` in `ci/*.json`, which is a users' manual in the same
way the old capability-letter table was (RFC 0008 §motivation). And a run
that fails answers "failed" with nothing behind the word — no log, no
failing step, no way to reproduce.
## Architecture: three layers
[architecture]: #architecture-three-layers
Each piece lives where somebody else would also use it:
| Layer | Owns | Why there |
|---|---|---|
| **dagnats core** | run engine, retries, approval gates; the facilities: log lane (#624), lifecycle events (#625), per-step timestamps (#626), scoped/JIT worker tokens (#627), worker directory (#628), run labels and filters (#629), re-register hash (#630), queue-depth signal (#632) | every dagnats front-end needs them; logs and timestamps can only be captured inside the engine and worker SDK |
| **dagnats CI module** (mounted in `dagnats serve`, #633) | `POST /v1/ci/compile` and `/v1/ci/validate` (the ci.yml compiler, errors as data); the worker set (dagger caller, Dagster adapter, later others); the annotation convention (#630) | forge-agnostic CI vocabulary — the module's GitHub front-end uses the identical compiler and workers; none of it knows what a landing is |
| **quarry `internal/ci`** | triggers from quarry events; tip-pair premerge orchestration; the checks panel and accept gating; `check.report` and `ci.trigger` ops; the Runners page; verdict-tag policy | quarry semantics — landings, protected branches, the ledger's rules |
Quarry speaks one URL. The compiler is never vendored and never shelled:
compilation happens at the engine's own version behind the same base URL
(`--dagnats`), so spec/engine skew is impossible by construction. The one
quarry-shaped piece that runs on workers — the premerge checkout step —
ships from the quarry repo, because its semantics (tip pairs, read-only
credentials, verdict keying) are the forge's.
## The spec
[spec]: #the-spec
A repository opts into CI by committing a file. Two formats, one truth:
- `ci/*.json` — raw dagnats `WorkflowDef`, honored exactly as today. The
engine format is the truth; everything else compiles to it.
- `ci/*.yml` — the small spec, compiled by the CI module at trigger time:
```yaml
checks:
test: { call: "test" }
lint: { call: "lint" }
build: { call: "build", needs: [test, lint], timeout: "20m" }
```
Rules, stated once and enforced by refusing to grow:
1. **No logic in the spec.** No expressions, no conditionals, no templating.
A step names a task, its needs, and its limits; logic lives in the tasks,
which is what a DAG engine is for. This is the trap that ate the
incumbent's format, avoided by construction.
2. **Validation is a surface, not a failure mode.** The settings page and
the empty CI page call `/v1/ci/validate` and render errors as data —
line, field, message — before anything runs. A repository with a spec
that does not compile shows the error where the runs would be, not a
silent absence.
3. The empty CI page offers a starter file, the way the ticket editor
offers its vocabulary.
## Triggers
[triggers]: #triggers
Four ways a run starts; the first exists today.
1. **Check-ins** — the watermark walk (`internal/ci/trigger.go`), unchanged
in shape: new check-in, read the pipeline files at that check-in,
compile if yml, start. Run input carries repo, check-in, branch, user,
comment, clone URL — and, new, **labels** (`repo=`, `checkin=`,
`branch=`) once dagnats #629 lands, so runs are findable and
cancellable by what they are about.
2. **Landings** — opening or updating a landing runs the branch's
pipelines on its merge with the target ([below](#premerge)). A new push
to the branch cancels the stale runs by label (#629) and starts fresh
ones.
3. **By hand** — a `ci.trigger` op: run a named pipeline now, with inputs.
Deploys, nightlies-on-demand, re-runs. Gated by `i` (RFC 0004: the
check-in letter — if you may write code here, you may run its
pipelines); the button on a red check row says "Re-run".
4. **From outside** — dagnats' own cron and external trigger envelopes
cover schedules and foreign events; and the loop quarry already
half-owns closes the fully-external case: outbound webhooks
(`landing.opened`, check-in events, RFC 0005) are what a foreign CI
subscribes to, tests, and answers via [`check.report`](#external-ci).
A JSON pipeline whose top-level `"quarry_when"` is `"landing"` (quarry's
key, namespaced apart from any engine field; the engine ignores it) runs
only where a proposal is judged: check-ins on the default branch and
pushes to a branch with an open landing. A ci.yml carrying the key is
refused, not silently run everywhere. A bare branch push
skips it; a re-run by hand runs it. When a landing opens on a tip that
lacks a verdict for a required check, the open starts only the pipelines
with no verdict there, so a landing opened after its push gets its
`"quarry_when": "landing"` pipelines once, without re-running the rest; this
needs the pipeline among the target's required checks. quarry's own race
detector (`ci/race.json`) is one, being the slowest check and one only a
proposal needs.
## Verdicts on landings
[verdicts]: #verdicts-on-landings
The Gerrit shape: the machine's opinion is a named thing beside the human's
decision, never mixed into it.
- **The checks panel.** Every landing shows one row per pipeline relevant
to it: status chip, pipeline name, started/duration, a link into the run
(quarry's log viewer once it exists; the dagnats console meanwhile).
External checks ([below](#external-ci)) render in the same panel, their
target URL standing in for the log link. Live via the SSE morphing the
CI page already uses; the no-script baseline is load-time state plus
refresh (RFC 0001 Principle 3).
- **Required checks** configure per branch, beside RFC 0007's protection
rules and gated the same way (`a`): a list of check names that must be
green before an accept proceeds. The vocabulary is names, not runs — a
required check with no run yet reads *waiting*, and blocks.
- **The accept button says why.** Green: "Accept — checks green". Red or
waiting: the control is disabled with the reason in words, exactly as
protected branches already refuse — *"test is failing"*, *"build has not
run"*. A control that would fail is not offered; one that names why is
better than none (RFC 0006 §strata-UI, the rule landings already live by).
- **Override is the owner's, said out loud.** A branch owner (RFC 0007) or
an `a`-holder may accept over a red or missing check only with a stated
reason, which is recorded as a landing comment before the accept is
written. The record shows both the red check and the sentence that
overrode it.
- **Auto-accept when green** is a per-landing opt-in by someone who could
accept it now were it green: the forge accepts on their behalf when the
required set turns green, attributing the accept to them with an
"auto-accepted: checks turned green" note. The freshness rule below
applies — a moved target re-arms the wait rather than accepting stale
evidence. This is the seed that grows into a merge queue; the queue data
structure itself is deliberately deferred.
## Premerge: nothing enters the DAG
[premerge]: #premerge-nothing-enters-the-dag
A landing's checks run on its merge with the target, the way a GitHub
pull request's checks run on its test merge (quarry#290). There is no
setting: every landing is tested this way.
The ledger is append-only, so the git-world trick — materialize a
speculative merge ref, test it, garbage-collect it — is exactly wrong here.
The merge is a tree the forge serves, never a check-in:
1. The runner fetches `/archive/pair:<branch tip>:<target tip>` exactly as
it fetches a check-in's archive. The forge probes the pair, merges it in
a scratch checkout opened against its own store, and serves the tree;
it never writes to the store. A pair that does not merge answers a
`merge_conflict` refusal naming the files, which fails the step once,
without the checkout retries: the conflict is the verdict.
2. The pipeline runs against that tree. **Nothing is ever committed.** The
runner needs no zit, no clone and no credential that could push.
When zit can answer this itself (danmestas/zit#223, `zit merge-tree`), the
forge's archive builder is the only thing that changes.
**Verdicts attach to the pair of tips** `(branch, target)`, never to a
check-in hash: the merge check-in does not exist yet, and the one accept
eventually writes will hash differently.
- **When runs start**: after every fold, each open landing whose branch tip
has no pair yet gets one, merged into the target as it is: that is a
landing opening and a push to its branch. The pipelines are the branch
tip's. A `"quarry_when": "landing"` pipeline runs on the default branch
and in landings' merges, never on a bare branch tip, as a
`pull_request` workflow runs only on pull requests.
- **When the target moves**: nothing re-runs, as nothing re-runs on a pull
request when its base moves. The landing says which target tip its
checks merged into and how far the target has moved since; the accept
is allowed on that verdict, and its merge message names the target tip
the checks tested. The next push to the branch tests the merge with the
target as it is then, and Re-run does so on demand.
- **External checks** reported on the branch tip count for its pairs, as a
pull request shows the statuses on its head commit.
- **Carry-forward**: after accept writes the real merge from the same pair,
the verdict may be tagged onto it where [tags are opted in](#verdict-tags)
— the portable record says this merge was tested before it existed. It
applies only when the pair's target tip is the one the merge landed on.
- **Strata**: landing a stack premerges rungs sequentially in one worktree;
the whole-stack verdict falls out of the same mechanics.
Rejected: speculative store copies (a real check-in hash, but a useless
one, at full-store cost) and test-after-accept with auto-revert (reverts
are ledger pollution, and the point is that trunk never goes red).
## Logs
[logs]: #logs
Split at the engine's edge (#624, as rescoped there): dagnats owns the
hot lane and the contract; retention belongs to the consumer — a forge
that needs a verdict to stay explainable for years stores the logs next
to the verdict.
- **Hot lane (dagnats)**: the worker SDK's `LogOut`/`LogErr` writers
stream step output as bounded chunks (`{seq, ts, stream: out|err|marker,
data}`) on the `BUILD_LOGS` JetStream stream, `logs.{runID}.{stepID}`;
a failure emits a `marker: failed` chunk so first-failure is recorded,
not inferred. `GET /runs/{id}/logs?step=&after_seq=&follow=1&from=failure`
reads and tails it (SSE on follow). Short TTL; the stream is a
documented consumer contract beside #625, and the terminal `run.*`
event is the drain trigger.
- **Retention (quarry)**: on a run's terminal event — or the poller
noticing, which is the same fact later — quarry drains the run's logs
and stores them beside its own `ci_run` record, one object per run in
the derived-data directory, with an S3-compatible target as
configuration for fleets that want the objects elsewhere. Cold reads
come back through the same viewer; it never knows which side answered.
Per-step durations come from #626 (shipped: the engine persists
`StartedAt`/`CompletedAt`).
Quarry's viewer is server-rendered chunks plus an SSE tail, with the spine
that makes logs bearable: collapsible per-step groups (`<details>`), the
failed step first, jump-to-first-failure, a permalink to any line — and on
every failure a printed **repro line** (`zit clone … && …`: the failing
task, locally, no forge in the loop), which is this architecture's answer
to SSH-into-the-build-VM. Failures that carry the annotation convention
(#630) additionally pin onto the landing diff as file/line notes.
## Runners
[runners]: #runners
A **Runners** section at both scopes — `/fleet/settings` and repository
settings — gated by RFC 0008's letters (`s` fleet, `a` repo):
- **Mint a token** (#627), scoped to task types or queues, labeled with who
and where. The page shows the one-liner to start a worker with it. Revoke
from the same list.
- **See what is attached** (#628): worker id, task types, hostname,
transport, liveness.
Each repository has a **worker group** — `ci.GroupFor(name)`, its own
name folded to one subject token — and a token minted from its Runners
page is scoped to that group alone. A group-scoped token cannot reach
another group or the ungrouped queue, so the group is the boundary and
the mint form asks for a label and nothing else. The fold matters twice:
a dot would make the group ambiguous with a dotted task type (the
dagnats#695 bypass), and `=` is the sentinel dagnats encodes a group with
(`task.{task}.={group}.{runID}`), so no repository name can spell a group
boundary of its own.
Because the spec lives in the owner's repository, the author chooses the
`worker_group:` their steps publish to — which makes the spelling in the
file an authorization decision. `ci.pipeline.save` refuses a group that
is not this repository's own, on the validate path as well as the save
path. The risk it answers is not reading another repository's work but
writing into it: a spec naming another group publishes its steps onto
that repository's machines.
Hosted provider fleets — the Blacksmith class — attach through the same
door, which is the point: a provider integrates once with dagnats and
serves every dagnats front-end. Their two extras are upstream: single-use
JIT registrations (#627) so a fresh VM per job never holds a reusable
credential, and the queue-depth signal (#632) to size fleets without
polling. A provider-run job is indistinguishable from a basement machine:
same logs, same steps, same verdicts.
## External CI
[external-ci]: #external-ci
For CI quarry does not run — a Jenkins, a bespoke script, a mirror's
pipeline — one op closes the loop:
- **`check.report`**: check-in hash *or* tip pair, a context name, a state
(`pending` / `success` / `failure` / `error`), a target URL, an optional
summary. External checks render in the checks panel (target URL as their
log link), count toward required checks when an administrator names
them, and earn verdict tags only where tags are opted in.
- Credentials are **reporter tokens**: minted and revoked on the Runners
page, scoped to `check.report` alone. This is not a second permission
system (RFC 0008 retired that): a token *authenticates*, and
authorization still asks the effective capability set — a reporter
token's set is simply fixed and one op wide. A compromised one can lie
about a check (visible, attributable, revocable) and do nothing else.
Subscribe (webhooks out) → test (their infrastructure) → report
(`check.report`) is the entire bring-your-own-CI contract.
## Verdict tags
[verdict-tags]: #verdict-tags
Pass/fail as tag artifacts is quarry's own invention — CI history that
clones with the repository — and it is also a write into the DAG on every
run. So it is **opt-in per repository**: a Settings toggle, with the fleet
default page saying what new repositories start as. Preserve mechanics:
- Tags already written are ordinary immutable artifacts. Toggling off stops
new writes and never shuns old ones.
- Toggling on resumes from the next terminal run; there is no backfill.
- The forge's `ci_run` table keeps full history either way; the tags are
the portable subset. Premerge verdicts reach the DAG only via
carry-forward at accept, since their subject does not exist before it.
The `--ci-verdicts` flag becomes the fleet default this setting reads.
## Capability mapping
[caps]: #capability-mapping
Per RFC 0008, every surface is letters, stated here once:
| Surface | Gate |
|---|---|
| See runs, checks, logs | the repository read gate, as `ci.list` today — CI is not more secret than the code |
| `ci.trigger`, re-run | `i` |
| Required-checks config | `a`, beside the RFC 0007 rules |
| Runners page, tokens (repo / fleet) | `a` / `s` |
| `check.report` | a reporter token, scoped to the op |
| Accept over red, auto-accept arming | whoever could accept the landing (RFC 0007 owners; the override additionally requires the stated reason) |
## Build order
[build]: #build-order
| Rung | Work | Waits on |
|---|---|---|
| 1 | This RFC | — |
| 2 | Checks panel on landings; required checks beside RFC 0007; disabled-accept wording; override-with-reason. Runs associate with a landing by its branch-tip check-in until rung 5's labels | nothing (status via today's poller) |
| 3 | `check.report` + reporter tokens; `ci.trigger`; the external loop documented | nothing |
| 4 | Spec: yml detection in the trigger, compile/validate via the CI module; starter file; errors-as-data on the settings and CI pages | #633 (#631 folded in) |
| 5 | Landing triggers; labels on every run; cancel-superseded; events replace the poll | #625, #629 |
| 6 | Runners page, both scopes | #627, #628 |
| 7 | Log viewer (tail, groups, first-failure, permalinks, repro line, diff annotations) | #624, #626, #630 |
| 8 | Premerge: the checkout step helper, per-branch mode, tip-pair verdicts, carry-forward, auto-accept-when-green | nothing upstream |
| 9 | Provider polish: JIT tokens surfaced, queue-depth on the fleet page | #627 ext, #632 |
Rungs 2 and 3 ship on today's engine with no upstream movement; that is
deliberate — the accept gets its evidence first, everything else makes the
evidence better.
## What this is not
[not]: #what-this-is-not
- **Not GitHub Actions compatible — yet, and never at the protocol level.**
Workflow-file compatibility (an `act`-executing worker mapping quarry
events to `push`/`pull_request`) is a plausible later module in the CI
layer, explicitly **deferred** from this RFC: the native spec ships
first, and any shim will be labeled a migration bridge with its fidelity
limits stated. The GHA *runner protocol* — impersonating GitHub's
Actions service so its runner binary or GHA-only farms attach — is out
forever: enormous, undocumented, churning, and even the forges famous
for Actions compatibility built their own runner instead. Providers get
our documented door, which is smaller than the one they already passed
through.
- **Not a merge queue** — yet. Tip-pair freshness plus auto-accept-when-
green covers the single-landing case; a batching, bisecting queue is a
later RFC if contention ever demands it.
- **Not an artifact or cache service.** Build artifacts and caches are the
workers' business; a run's durable outputs are its logs and its verdict.
- **Not post-accept enforcement.** Nothing here reverts, force-moves, or
edits history on a red check after the fact; the gate is before the
merge, and override is a human with a reason.
## Testing
[testing]: #testing
The harness grows a fake engine (an httptest server speaking `/workflows`,
`/runs`, `/v1/ci/*`) so every rung is testable without NATS: required
checks block and unblock the accept control; the override records its
comment before the accept; a moved target leaves a pair verdict standing
and is named on the landing; `check.report` rows render and count; tag
opt-in writes on terminal runs and toggling off writes nothing. Building
a pair's merged tree never writes to the store, and its test against a
real store asserts the store file is byte-identical before and after a
clean merge and a conflicting one.