Skip to content
quarry Artifact [f4e9a3fec6]
Login

Artifact [f4e9a3fec6]

Artifact f4e9a3fec66ebbad0420850bb279c12f1c23b5ab709199bdf185caf72b25ddf6:


Summary

Setting up CI for pikchr-studio, the second repository on zeitforge to get it, took two JSON files to say one thing, timeouts in nanoseconds, a retry block whose zero means "run once", a worker group typed by hand, task names that had to start with the repository's name (learned from a 403), and a dedicated runner with a production token in a root-only file on the box, guarded by a flag written that afternoon. Every one of those is a fact about the engine that the author had to learn.

Codeberg's Woodpecker, GitHub Actions, GitLab and sourcehut all converge on the same shape. This RFC adopts it in GitHub Actions' words, the ones most people already know:

Motivation

The deploy question showed the gap. "Deploy pikchr-studio after a landing" needed a new pipeline key (quarry_when: default-branch), a new runner flag (--branch trunk), a new system user, and a second worker token, and it still only works for trunk. The owner asked the right question: why only trunk, and not wherever the user says?

The answer every other forge gives has two halves, and quarry had only been building one:

  1. The file says where a job wants to run. It is read at the check-in under test, so a branch can rewrite it. It is a convenience.
  2. Settings say where a secret may go. A branch cannot edit settings. This is the safeguard.

GitHub puts a deploy's branch policy on the environment, in settings; GitLab exposes protected variables to protected branches only; Woodpecker limits each secret by event and keeps it from pull requests by default. quarry already has the right primitive for the second half: a protected branch (RFC 0007) is one a check-in reaches only through a landing somebody accepted. "Protected branches only" is therefore "reviewed code only", configured where the owner already configures who may land.

The file

ci/<name>.yml, one workflow per file, as RFC 0009 already named it. The keys are GitHub Actions' where Actions has a key for the idea, because that is what people will copy from and search for. Nothing promises that an Actions workflow runs unchanged; see What this is not.

pikchr-studio's two workflows, as they would be written:

# ci/test.yml: every push and every landing
on: [push, landing]
jobs:
  test:
    timeout-minutes: 30
    env:
      PIKCHR_STUDIO_DIAG: "1"
    steps:
      - run: npm test
  worker:
    steps:
      - run: npx --yes [email protected] deploy --dry-run --outdir dist-dry-run
# ci/deploy.yml: publish after a landing
on:
  push:
    branches: [trunk]
jobs:
  test:
    steps:
      - run: npm test
  deploy:
    needs: [test]
    environment: production
    steps:
      - run: npx --yes [email protected] deploy
        env:
          CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}

What the author no longer writes, and where it went:

Today Now
ci/runner.json naming each task's argv a job's steps:, each run: a shell script run with set -e, in the job's working copy
task type pikchr-studio.test derived: quarry names the task from the repository and job
"worker_group": "pikchr-studio" derived: always the repository's own group (RFC 0009 already refuses any other)
"timeout": 1800000000000 timeout-minutes: 30; a default applies when absent
"retry": {"max_attempts": 0} the default is once; retries: 2 when wanted (quarry's key, Actions has none). In a JSON pipeline max_attempts is the total attempts, first run included: 2 runs a step twice, 0 and 1 run it once, and the run page's "attempt N of M" shows the number written. quarry hands the engine one less, since dagnats counts retries
"quarry_when": "landing" / "default-branch" on:
"_comment" YAML comments

Jobs without needs: run in parallel, as in Actions; a job's steps run in order and stop at the first that fails. name: on a job or a step labels it in the checks panel and the log viewer; working-directory: and env: work at workflow, job and step level with the nearest winning, as in Actions.

on:

At the top of the file, for the whole workflow. Actions' shapes are accepted: a single event (on: push), a list (on: [push, landing]), or a map with filters.

Event Starts the workflow on Filters
push a check-in on a branch branches, branches-ignore, paths, paths-ignore
landing a landing's tested merge (RFC 0009 §premerge); quarry's pull_request branches (the target), paths (every file the landing changes)
tag a tag added to a check-in, once, on that check-in (RFC 0020 §releases) tags, tags-ignore; version-* when neither is given
workflow_dispatch a Re-run or ci.trigger by somebody holding i none
schedule a cron schedule (- cron: "17 3 * * *"), on the default branch none

Absent on: means [push, landing], today's behaviour. A tag is a symbolic tag added to a check-in (+sym-NAME, or a propagating *sym-NAME, counted once on the check-in that carries it); a branch's own sym- name and raw tags such as closed are not tag events. Branch and path filters are globs with Actions' matching (*, **, ! to negate in a list). A job cannot narrow events on its own: a workflow that deploys from trunk and tests everywhere is two files, which is how Actions users already write it without if:.

Deliberately absent: if: and every ${{ }} expression except ${{ secrets.NAME }}. RFC 0009 rule 1 stands: filters decide when, steps decide logic. A job that needs a real condition runs a step that exits early. Writing if: or any other ${{ }} is an error that says so, never a silently ignored key.

quarry_when in a JSON pipeline keeps working and is documented as the engine-format spelling of on: [push, landing] minus bare pushes, and of on: {push: {branches: [<default>]}}.

Errors

Unknown keys are errors with a line and a column; so are a needs naming no job, a cycle, a ${{ secrets.NAME }} naming no secret the job can reach, an environment: the repository does not have, an event quarry does not emit, and any Actions key quarry does not implement (uses:, runs-on: beyond the ignored self-hosted, strategy:, services:, container:), each named as not supported rather than as a typo. They show on the CI page where the runs would be and on the landing's checks panel, per RFC 0009 rule 2.

Environments and secrets

An Environments section in repository settings, gated by a (RFC 0008). Each environment has a name, secrets, and a branch policy:

Secrets outside any environment are repository secrets, available to every job, and they too reach protected branches only; the owner can widen a repository secret to every push when it is not sensitive (a public API key, say). A job's environment: adds that environment's secrets, which shadow a repository secret of the same name.

Values are write-only in the UI: set, replace, delete, never shown. The settings trail (as for every setting) records who changed which secret or policy and when, never the value.

Delivery never goes through the engine. dagnats persists a run's input, so a secret in it would sit in NATS storage. Instead the runner, when it claims a job whose steps name secrets, asks the forge's bridge for them with its worker token. The forge answers only if the job's check-in qualifies, decided the way --branch decides today: it reads the check-in's branch from its own index, not from the task input, and refuses a pair outright. The runner puts the values in the step's environment and masks them in the log it streams (every occurrence of a value is replaced before it leaves the machine). Masking is a courtesy, not the protection: a step can print a secret base64-encoded. The protection is that unreviewed code never receives one.

A runner's jobs must not share a Unix user

On zeitforge's box every repository's CI runs as one user, ciworker, and on Linux a process can read the environment of any other process of the same user (/proc/<pid>/environ). A secret delivered to pikchr-studio's deploy job would be readable by a quarry branch test running at the same moment, and any job could read its runner's own environment, worker token included, and use it to ask for secrets for a task that runner holds.

As built (rung 5b). A runner started with --job-users qjob1,qjob2 runs every task as one of those users, one per running task, and only such a runner receives secrets; any other refuses a job that asks for one. The pool users are unprivileged, each with a group of its own, and the runner may become them through sudo and nothing else (ci-user ALL=(qjob1,qjob2) NOPASSWD: ALL); nothing runs as root. The runner's user belongs to each pool group, and a job's checkout is shared with that group alone (group-owned, setgid, nothing for others), so concurrent jobs can neither read nor touch each other's trees. The job's environment, secrets included, reaches it over stdin to quarry job-env, never on a command line, which every user on the machine can read; it holds nothing of the runner's but PATH and LANG. After the job, the runner ends whatever the pool user left running and reclaims the tree.

DynamicUser= was the first idea and was set aside: a one-shot user gets its id only at start, so it cannot own the checkout it works in without making that tree writable by everyone, which would let a concurrent job tamper with a deploy's files.

The runner itself needs its own user when it receives secrets: a second runner's jobs running as the same user could otherwise read its environment and its token. On zeitforge that means the pikchr-studio runner runs as its own system user, not ciworker.

This retires the dedicated deploy runner. A runner holds a worker token and nothing else; quarry runner --branch stays as an extra lock for owners who want a machine that runs one branch, but it is no longer how a secret is protected.

Where each piece lives

RFC 0009's three layers, unchanged in shape:

Layer Does
dagnats CI module parses and validates the YAML (jobs, steps, run, needs, timeout-minutes, env, environment, secret references by name), compiles it to a WorkflowDef, and returns the workflow's on: as structured data it does not interpret
quarry trigger evaluates on: against quarry's events (it is the only one that knows what a landing or a protected branch is) and launches the matching workflows
quarry runner runs a job's steps from the task input; fetches and masks secrets; runs a secret-holding job under its own user; still honours ci/runner.json for JSON pipelines
quarry forge the Environments page, storage, the bridge endpoint that decides delivery

on: stays uninterpreted in dagnats on purpose: its GitHub front-end has its own events, and a forge-agnostic compiler that knew "landing" would not be. dagnats' current checks:/deploy: spec keeps compiling beside the new one.

Build order

Rung Work Waits on
1 This RFC done: questions settled
2 dagnats: the Actions-shaped spec in the CI module (on, jobs, needs, steps/run, timeout-minutes, env, environment, ${{ secrets.NAME }} references), on passed through as data, line/column errors, unsupported Actions keys named a dagnats issue
3 quarry runner: a job task type that runs steps from the task input; masks a list of values in its log sink; runs a job under a given user nothing
4 quarry trigger: compile ci/*.yml, evaluate on:, map quarry_when onto it; errors on the CI page and checks panel rung 2
5 Environments and secrets: settings section, encrypted storage, trail, the bridge endpoint with the delivery rule, runner fetch, masking and per-job user rung 3
6 Starter files on the empty CI page (test; test plus deploy); docs/running.md and a how-to rung 4
7 Move quarry's own CI and pikchr-studio to YAML; pikchr-studio's deploy uses a production environment; stop the interim deploy runner and revoke its token rungs 4, 5

Rungs 2 and 3 are independent and can go in parallel; rung 5 can start beside rung 4.

Rung 5's storage

Secrets are encrypted at rest with a key the forge reads from its environment (not from the database it encrypts), so a copied database or backup does not carry usable secrets. A backup restored without the key restores the environments, names and policies, and every value must be set again; the Environments section says so when it happens.

What this is not

Settled questions

Decided by the owner on 2026-09-29:

  1. YAML, over TOML. None of Woodpecker, Actions, GitLab or sourcehut uses TOML, people copy CI files between projects, and the nested filters read worse in TOML. YAML's traps are answered by strict parsing.
  2. GitHub Actions' words, over Woodpecker's (steps, commands, when) and dagnats' current (checks, deploy). The most people know them, and their environments are exactly the deploy answer. The cost is the expectation of compatibility, answered by naming every unsupported key in its error.
  3. ci/, over a dot-directory. RFC 0009 and every repository with CI today already use it.

Testing