Skip to content
quarry Running quarry
Login

Running quarry

Two ways: the compose stack, or by hand. The hand-rolled path is what this was developed against and is what the compose file encodes, so it is written down first — a run book that stays true even if the images are never built.

What the stack is

piece role state
zit the store. quarry shells out to it for every read and write (0001 §2) the .zit files themselves
quarry the forge: indexes, UI, API, sync fronting derived indexes, embedded NATS, OLTP database
notify notifications: workflows, subscribers, topics, the inbox quarry embeds SQLite
dagnats the workflow engine notify runs on: embedded NATS + JetStream + an HTTP bridge JetStream data

zit is not a runtime dependency in the ordinary sense — it is quarry's only store interface, so its version is part of quarry's behaviour. On 2026-08-16 three zit builds changed what quarry does: frames got 31% faster, a card-parsing fix changed which artifacts parse at all (and until it landed, sqlite's 794 tickets were invisible), and a new stat -R let quarry serve a bare store. Pin the pair; do not let a deploy resolve zit from whatever is on PATH.

By hand

dagnats

Runs under launchd on this machine as com.dmestas.dagnats, binding NATS on 4222 and its bridge on 8080, with data in ~/Library/Application Support/dagnats.

launchctl kickstart -k gui/$(id -u)/com.dmestas.dagnats

A freshly-built Go binary replacing it must be ad-hoc signed or launchd kills it with OS_REASON_CODESIGNING and the service silently fails to come back:

go build -o ~/go/bin/dagnats ./cmd/dagnats
codesign -s - -f ~/go/bin/dagnats
launchctl kickstart -k gui/$(id -u)/com.dmestas.dagnats

In standalone mode dagnats binds loopback, which is why anything talking to it shares its network namespace under compose rather than reaching it by service name.

notify

notify bootstrap -dev -db notify.db -org quarry -env dev   # prints apiKey once
notify serve -dev -db notify.db -addr 127.0.0.1:3000 \
  -nats-url nats://127.0.0.1:4222 -bridge-url http://127.0.0.1:8080
curl -s localhost:3000/readyz     # {"status":"ok","bridge":true,"db":true}

-dev derives a fixed, publicly-known master key. Outside development set NOTIFY_MASTER_KEY_HEX.

Provisioning is quarry's job, not an operator's (RFC 0011 §7). What used to be a page of curl here is now code: quarry creates a notify subscriber when an account is added, adds or removes the account from a repository's repo-<name> activity topic when it is granted, revoked, or subscribes to a thread, and creates its six workflows (quarry-mention, quarry-forum-reply, quarry-activity, quarry-landing, quarry-error, quarry-slow — defined in internal/notify/workflows.go) at startup, leaving any workflow an operator already edited untouched. quarry-landing is provisioned ahead of need: nothing triggers it yet, that lands with the pull request review surface. All of it is best-effort: a notify outage never fails the account or grant op that raised it.

Upgrading a forge provisioned before RFC 0011 Phase 3. The workflows now carry more than one step each: quarry-mention, quarry-forum-reply and quarry-landing gained a push step after their in-app one, and quarry-activity gained a 15-minute digest before its in-app step (RFC 0011 §6). Because quarry never PUTs over a workflow that already exists — an operator may have edited its copy, and notify's idempotent upsert (notify#303) has not landed — an existing install keeps the single-step definitions it was provisioned with. A forge provisioned for the first time needs nothing.

To pick the new definitions up, delete those four workflows once and restart quarry; the next startup recreates them from internal/notify/workflows.go. The {slug} is the trigger identifier, not the workflow's ULID:

K=...   # the notify apiKey from bootstrap
for w in quarry-mention quarry-forum-reply quarry-landing quarry-activity; do
  curl -fsS -X DELETE -H "Authorization: ApiKey $K" \
    http://127.0.0.1:3000/v1/workflows/$w
done
# then restart quarry; startup recreates all six.

Deleting a workflow deletes nothing a person can see — messages already delivered stay in their inbox. Do it during a quiet moment anyway: a trigger fired between the delete and the restart reaches nobody.

What degrades until notify#305 lands. notify has no PUT /v1/subscribers/{id}/preferences and no webpush credential DELETE yet. quarry ships into that and says so rather than pretending:

Both clear themselves once the routes exist — no migration, no reset.

quarry

quarry serve --addr 127.0.0.1:8950 --data ./data \
  --notify http://127.0.0.1:3000 --notify-key "$K" \
  --repo quarry=$HOME/quarry

--data is not optional in practice: it defaults to .quarry beside the first --repo, which writes derived state into whichever repository happens to be listed first. Always pass it.

Gates are open until they are closed, and closing them is two separate credentials on purpose — accounts are the forge identity (argon2id), the sync secret is the RFC §6.3 value the protocol fixes:

quarry account add --user you --admin --data ./data   # closes web/API writes
quarry user add --repo name=$HOME/quarry --user you --data ./data   # closes push

The account gate is global: the first account closes writes on every repository in the fleet, not just one.

The attention stream and HTTP/2

Signed-in pages open one Server-Sent Events connection for the attention stream — the inbox badge and the count in the tab title (RFC 0011 §5). In-page banners are off by default and are turned on per account on /inbox ("In-page banners"), which is also where desktop notifications are opted into; with both off the stream still moves the badge and the title, and nothing ever covers the page being read. On plain HTTP/1.1 a browser allows six connections per origin, and a repository page already spends one or two of them on the chat and timeline streams. So the default opens the attention stream on repository pages only: a stream on every page would compete with the page it sits on for the same six slots, and the symptom is a page that stalls halfway through loading rather than an error anybody can read.

With HTTP/2 in front there is no such limit — one connection carries every stream — so the whole forge can hold it open:

quarry serve ... --attention-everywhere

Use it when quarry is behind TLS termination that negotiates h2, or behind a proxy speaking h2c to a browser. quarry serves plain HTTP/1.1 itself, so the flag is a statement about what is IN FRONT of it and quarry cannot check it: turned on without h2, it is the six-connection problem on every page.

Nothing else changes. Without the stream a page still renders its own state and the inbox badge is fetched once after load, which is the Principle 3 behaviour every one of these surfaces keeps.

The activity robot

A repository can have its timeline talk in its chat channel (RFC 0011 §10): settings → Activity robot, in the same event-pattern syntax webhooks take (*, a noun like checkin, or an exact kind like landing.accepted). Empty turns it off. Robot lines never count as unread and never mention anybody.

Fleet admin

/fleet is the operator page — every store's ingest health, and the upstreams it mirrors. It is site admin only: admin on one repository is not enough, because the page reports operational detail across the whole fleet and can force writes into any of it.

Health is per-process. Total and Fallbacks count what THIS server has ingested since it started, so a restart with a warm index reports zero of both however much the store holds. The parse-fallback number worth looking at is the one a quarry reindex produces.

Mirroring

A mirrored store tracks an upstream: quarry pulls, and never pushes.

curl -u you:pw -X POST http://localhost:8080/api/v1/mirrors \
  -H 'Content-Type: application/json' \
  -d '{"repo":"fossil","url":"https://fossil-scm.org/home","every":3600}'

every is seconds and is floored at 60 — these upstreams are other people's servers, and the floor is a courtesy the configuration surface cannot opt out of. --mirror-tick (default 1m) is how often due-ness is evaluated, not how often anything is pulled.

Pulls take the store's single-writer lease, so mirroring is safe with more than one node serving the same store. A pull that brings nothing new does not trigger a re-index and is not logged; almost every one is quiet, and a line per store per interval buries the ones that matter.

Two limits worth knowing before you rely on it:

With compose

export NOTIFY_SRC=../notify           # a notify checkout; it builds dagnats too
export ZIT_REF=<commit>               # pin the store implementation
docker compose up --build

Then bootstrap notify once (docker compose run --rm notify bootstrap …), put the key in QUARRY_NOTIFY_KEY, and create the workflows and topics as above.

docker run <image> cat /etc/quarry-build reports which quarry and which zit an image carries, without starting it.

Unverified. Dockerfile and compose.yml have not been built — no container runtime is installed on the development machine. The two build recipes inside them are verified natively: zit compiles from a clean clone of the GitHub mirror with zig build --release=fast, and quarry builds fully static with CGO_ENABLED=0 (its SQLite is WASM-based and Pikchr runs on wazero, so nothing needs a C toolchain). What remains untested is the image assembly and the compose wiring.

Ports

port what
4222 dagnats NATS
8080 dagnats HTTP bridge (and its console — unauthenticated, keep it on loopback)
3000 notify API
8950 quarry

quarry's own embedded NATS binds nothing (DontListen), so it never collides.

Working with several agents on one store

Several people — or several agents — committing to one .zit is the ordinary shape here, and 2026-08-29 taught its failure modes in one afternoon. The rules that keep it sane:

rule why
Land through the forge — tools/quarry-land, the landing page, or the API; never zit landing accept required checks, override-with-reason, auto-land and supersede exist only in the forge's write path. A store-level accept walks past all of them, and the store cannot refuse it — twice in one day, red-checked and unformatted work reached trunk that way
Arm auto-land at open (quarry-land <repo> <id> --when-green) nothing merges red out of impatience, and nothing waits on a human who is actually a busy agent
Prefix branches per effort (attention-*, ci-*, resilience-*) supersede-cancel, filters and the eye can all scope to whose work is whose
Commit or stash before leaving a shared checkout an uncommitted edit left in a serving tree turned a routine zit update into a merge conflict mid-deploy
Deploy only via the sanctioned script, and never build from a tree another agent may be editing the deploy script's job is to be the one honest path; a second, casual path is where the conflicts and truncated logs came from
Treat the zit binary as part of quarry's behaviour (see above) two zit releases in two weeks each silently broke shell-out seams. The CI worker runs a pinned zit (-zit); the TestZitContract canary names the next drift

The forge holds the gates; the store holds the truth; the agreement is that nobody asks the store to do the gate's job.