# Running quarry
Two ways: the compose stack, or by hand. The hand-rolled path is what
this was developed against and is what the compose file encodes, so it
is written down first — a run book that stays true even if the images
are never built.
## What the stack is
| piece | role | state |
|---|---|---|
| **zit** | the store. quarry shells out to it for every read and write (0001 §2) | the `.zit` files themselves |
| **quarry** | the forge: indexes, UI, API, sync fronting | derived indexes, embedded NATS, OLTP database |
| **notify** | notifications: workflows, subscribers, topics, the inbox quarry embeds | SQLite |
| **dagnats** | the workflow engine notify runs on: embedded NATS + JetStream + an HTTP bridge | JetStream data |
`zit` is not a runtime dependency in the ordinary sense — it is quarry's
only store interface, so **its version is part of quarry's behaviour**.
On 2026-08-16 three zit builds changed what quarry does: `frames` got
31% faster, a card-parsing fix changed which artifacts parse at all (and
until it landed, sqlite's 794 tickets were invisible), and a new
`stat -R` let quarry serve a bare store. Pin the pair; do not let a
deploy resolve zit from whatever is on PATH.
## By hand
### dagnats
Runs under launchd on this machine as `com.dmestas.dagnats`, binding
NATS on 4222 and its bridge on 8080, with data in
`~/Library/Application Support/dagnats`.
```sh
launchctl kickstart -k gui/$(id -u)/com.dmestas.dagnats
```
A freshly-built Go binary replacing it must be **ad-hoc signed** or
launchd kills it with `OS_REASON_CODESIGNING` and the service silently
fails to come back:
```sh
go build -o ~/go/bin/dagnats ./cmd/dagnats
codesign -s - -f ~/go/bin/dagnats
launchctl kickstart -k gui/$(id -u)/com.dmestas.dagnats
```
In standalone mode dagnats binds **loopback**, which is why anything
talking to it shares its network namespace under compose rather than
reaching it by service name.
### notify
```sh
notify bootstrap -dev -db notify.db -org quarry -env dev # prints apiKey once
notify serve -dev -db notify.db -addr 127.0.0.1:3000 \
-nats-url nats://127.0.0.1:4222 -bridge-url http://127.0.0.1:8080
curl -s localhost:3000/readyz # {"status":"ok","bridge":true,"db":true}
```
`-dev` derives a fixed, publicly-known master key. Outside development
set `NOTIFY_MASTER_KEY_HEX`.
**Provisioning is quarry's job, not an operator's** (RFC 0011 §7). What
used to be a page of `curl` here is now code: quarry creates a notify
subscriber when an account is added, adds or removes the account from
a repository's `repo-<name>` activity topic when it is granted,
revoked, or subscribes to a thread, and creates its six workflows
(`quarry-mention`, `quarry-forum-reply`, `quarry-activity`,
`quarry-landing`, `quarry-error`, `quarry-slow` — defined in
`internal/notify/workflows.go`) at startup, leaving any workflow an
operator already edited untouched. `quarry-landing` is provisioned
ahead of need: nothing triggers it yet, that lands with the pull
request review surface. All of it is best-effort: a notify outage
never fails the account or grant op that raised it.
**Upgrading a forge provisioned before RFC 0011 Phase 3.** The workflows
now carry more than one step each: `quarry-mention`, `quarry-forum-reply`
and `quarry-landing` gained a `push` step after their in-app one, and
`quarry-activity` gained a 15-minute `digest` before its in-app step
(RFC 0011 §6). Provisioning upserts with notify#303's
`PUT /v1/workflows/{slug}?onConflict=keep`, which creates a missing
workflow and leaves an existing one exactly as it is — an operator may
have edited its copy, and `keep` is what protects that edit from the
next restart. So an existing install keeps the single-step definitions
it was provisioned with. A forge provisioned for the first time needs
nothing.
To pick the new definitions up, delete those four workflows once and
restart quarry; the next startup recreates them from
`internal/notify/workflows.go`. The `{slug}` is the trigger identifier,
not the workflow's ULID:
```sh
K=... # the notify apiKey from bootstrap
for w in quarry-mention quarry-forum-reply quarry-landing quarry-activity; do
curl -fsS -X DELETE -H "Authorization: ApiKey $K" \
http://127.0.0.1:3000/v1/workflows/$w
done
# then restart quarry; startup recreates all six.
```
Deleting a workflow deletes nothing a person can see — messages already
delivered stay in their inbox. Do it during a quiet moment anyway: a
trigger fired between the delete and the restart reaches nobody.
**What degrades until notify#305 lands.** notify has no
`PUT /v1/subscribers/{id}/preferences` and no webpush credential
`DELETE` yet. quarry ships into that and says so rather than pretending:
- `me.prefs.set` writes its forge-local half first (the toast and
desktop switches, the per-repository mute) and reports
`applied.notify: false` when the notify half could not be stored. The
channel matrix a person set is then not yet in force.
- `me.push.unsubscribe` keeps the device row marked
`pending_remote_delete` instead of forgetting a browser notify still
pushes to, and retries on the next `me.push.subscribe`.
Both clear themselves once the routes exist — no migration, no reset.
### quarry
```sh
quarry serve --addr 127.0.0.1:8950 --data ./data \
--notify http://127.0.0.1:3000 --notify-key "$K" \
--repo quarry=$HOME/quarry
```
`--data` is not optional in practice: it defaults to `.quarry` beside the
**first** `--repo`, which writes derived state into whichever repository
happens to be listed first. Always pass it.
Gates are open until they are closed, and closing them is two separate
credentials on purpose — accounts are the forge identity (argon2id), the
sync secret is the RFC §6.3 value the protocol fixes:
```sh
quarry account add --user you --admin --data ./data # closes web/API writes
quarry user add --repo name=$HOME/quarry --user you --data ./data # closes push
```
The account gate is **global**: the first account closes writes on every
repository in the fleet, not just one.
### The attention stream and HTTP/2
Signed-in pages open one Server-Sent Events connection for the attention
stream — the inbox badge and the count in the tab title (RFC 0011 §5).
In-page banners are off by default and are turned on per account on
`/me/notifications`, which is also where desktop notifications are
opted into; with both off the stream still moves the badge and the
title, and nothing ever covers the page being read. On plain HTTP/1.1 a browser allows **six connections per
origin**, and a repository page already spends one or two of them on the
chat and timeline streams. So the default opens the attention stream on
**repository pages only**: a stream on every page would compete with the
page it sits on for the same six slots, and the symptom is a page that
stalls halfway through loading rather than an error anybody can read.
With HTTP/2 in front there is no such limit — one connection carries
every stream — so the whole forge can hold it open:
```sh
quarry serve ... --attention-everywhere
```
Use it when quarry is behind TLS termination that negotiates h2, or
behind a proxy speaking h2c to a browser. quarry serves plain HTTP/1.1
itself, so the flag is a statement about what is IN FRONT of it and
quarry cannot check it: turned on without h2, it is the six-connection
problem on every page.
Nothing else changes. Without the stream a page still renders its own
state and the inbox badge is fetched once after load, which is the
Principle 3 behaviour every one of these surfaces keeps.
### Push
`/me/notifications` carries the whole of RFC 0011 §6: the family x
channel matrix (mention, reply, landing, activity, by in-app, push and
email), the two forge-local switches, the repositories this account
muted, and **Push on this device**.
Push is the one feature in quarry that needs JavaScript, recorded as an
explicit exception in RFC 0011 §11. Enabling it registers `/sw.js`, asks
the browser for notification permission from the click that asked for
push, subscribes with notify's VAPID key, and posts the resulting
`PushSubscription` to `POST /me/push`. With JavaScript off the control
says so and nothing else on the page changes. On iOS a site must be
added to the Home Screen before Safari will grant push at all — the
control says that too.
Quarry never holds a VAPID key and never stores an endpoint: the
subscription goes to notify as subscriber credentials, and quarry keeps
a hash, a user-agent label and a timestamp so the page can list the
browsers somebody may want to turn off. That is also why a browser other
than the one being used is turned off **from that browser**.
What to expect until the upstream work lands:
- **notify#302** (the `webpush` provider) — subscriptions are accepted
and stored, but nothing is delivered by push yet. The in-app inbox,
the badge and the attention stream are unaffected.
- **notify#305** (preferences route, credential `DELETE`) — the banner
after saving preferences says `saved here; notify not reachable` and
the page reports `applied.notify` honestly; a device turned off shows
`removal pending` and is retried the next time push is enabled.
Neither needs a migration or a reset on this side.
### The activity robot
A repository can have its timeline talk in its chat channel
(RFC 0011 §10): **settings → Activity robot**, in the same event-pattern
syntax webhooks take (`*`, a noun like `checkin`, or an exact kind like
`landing.accepted`). Empty turns it off. Robot lines never count as
unread and never mention anybody.
## Fleet admin
`/fleet` is the operator page — every store's ingest health, and the
upstreams it mirrors. It is **site admin** only: admin on one repository
is not enough, because the page reports operational detail across the
whole fleet and can force writes into any of it.
Health is per-process. `Total` and `Fallbacks` count what THIS server
has ingested since it started, so a restart with a warm index reports
zero of both however much the store holds. The parse-fallback number
worth looking at is the one a `quarry reindex` produces.
### Mirroring
A mirrored store tracks an upstream: quarry pulls, and never pushes.
```sh
curl -u you:pw -X POST http://localhost:8080/api/v1/mirrors \
-H 'Content-Type: application/json' \
-d '{"repo":"fossil","url":"https://fossil-scm.org/home","every":3600}'
```
`every` is seconds and is floored at 60 — these upstreams are other
people's servers, and the floor is a courtesy the configuration surface
cannot opt out of. `--mirror-tick` (default 1m) is how often due-ness is
evaluated, not how often anything is pulled.
Pulls take the store's single-writer lease, so mirroring is safe with
more than one node serving the same store. A pull that brings nothing
new does not trigger a re-index and is not logged; almost every one is
quiet, and a line per store per interval buries the ones that matter.
Two limits worth knowing before you rely on it:
- **The store needs a working checkout.** `zit pull` refuses `-R` and
operates on a working directory, so a genuinely bare store cannot be
mirrored. Stores created by `zit clone` or `zit import-fossil` carry
one, so this bites only hand-built bare stores.
- **The upstream must allow anonymous clone.** Some Fossil servers serve
a browsable timeline while refusing sync — `sqlite.org/lemon` is one.
Note that `fossil clone` exits 0 on that refusal, so check for the
repository, not the exit status.
### Fossil chat bridge
An opt-in connector between one repository's chat channel and one
Fossil server's chat (RFC 0011 §9). It is configured per store, from
`/fleet` or through the op:
```sh
curl -u you:pw -X POST http://localhost:8080/api/v1/r/fossil/mirror/chatbridge \
-H 'Content-Type: application/json' \
-d '{"url":"https://fossil-scm.org/home","user":"quarrybot","password":"..."}'
```
Detach with `{"detach":true}`. The password is stored and is never
returned by any op or rendered on any page; the only reader is the
poller.
**The upstream needs two things**, neither of them a default:
1. A login holding fossil's `C` capability. Chat is gated on it.
2. `http_authentication_ok` in the repository's config, or fossil
refuses HTTP Basic and every request is anonymous. It is not a
`fossil setting`, so it goes in directly:
```sh
fossil sql -R /path/to/repo.fossil \
"INSERT OR REPLACE INTO config(name,value,mtime) \
VALUES('http_authentication_ok','1',now());"
```
Quarry polls `/chat-poll` with a nine-minute client timeout (fossil
hangs the request for `chat-poll-timeout`, 420 s by default) and backs
off from 30 s to 5 minutes while an upstream is failing. Every request
carries a same-origin `Referer`: without one fossil treats the poll as
CSRF-unsafe and refuses its own table setup, which surfaces as a
database error rather than as an authentication failure.
**What it does.** Messages from the upstream arrive in the channel as
`fossil:<user>` with `Origin: fossil`. A fossil deletion becomes a
tombstone on the message it removed. An attachment becomes a link to
the upstream's `/chat-download` — fossil keeps the file. Quarry's own
messages go out to `/chat-send` prefixed `**name:** `, because fossil
attributes everything to the login the bridge signs in as. The bridge
never forwards robot lines or lines that came from fossil, and never
re-imports its own login's messages.
**What it does not do.**
- **No edits.** A fossil message cannot be edited from quarry, and a
quarry edit is not carried upstream. Only deletions cross, and only
in the fossil-to-quarry direction.
- **No mentions.** Fossil chat has none, so a bridged line raises
nobody's badge and triggers no notification.
- **No account mapping.** `fossil:drh` is a name from another server's
user table, not a quarry account, and the prefix says so.
- **Two Fossil servers are two chats.** Fossil's chat is a
server-local table that never syncs, so bridging a mirror connects
you to that ONE server's conversation, not to the project's.
- **The window is still quarry's.** A bridged channel holds seven
days like every other; the bridge is a connector, not an import.
When this repository's activity robot is on, the upstream's
`chat-timeline-user` announcements are skipped — otherwise every
check-in is announced twice, once by each robot.
## With compose
```sh
export NOTIFY_SRC=../notify # a notify checkout; it builds dagnats too
export ZIT_REF=<commit> # pin the store implementation
docker compose up --build
```
Then bootstrap notify once (`docker compose run --rm notify bootstrap …`),
put the key in `QUARRY_NOTIFY_KEY`, and create the workflows and topics
as above.
`docker run <image> cat /etc/quarry-build` reports which quarry and which
zit an image carries, without starting it.
> **Unverified.** `Dockerfile` and `compose.yml` have not been built —
> no container runtime is installed on the development machine. The two
> build recipes inside them *are* verified natively: zit compiles from a
> clean clone of the GitHub mirror with `zig build --release=fast`, and
> quarry builds fully static with `CGO_ENABLED=0` (its SQLite is
> WASM-based and Pikchr runs on wazero, so nothing needs a C toolchain).
> What remains untested is the image assembly and the compose wiring.
## Ports
| port | what |
|---|---|
| 4222 | dagnats NATS |
| 8080 | dagnats HTTP bridge (and its console — unauthenticated, keep it on loopback) |
| 3000 | notify API |
| 8950 | quarry |
quarry's own embedded NATS binds nothing (`DontListen`), so it never
collides.
## Working with several agents on one store
Several people — or several agents — committing to one `.zit` is the
ordinary shape here, and 2026-08-29 taught its failure modes in one
afternoon. The rules that keep it sane:
| rule | why |
|---|---|
| **Land through the forge** — `tools/quarry-land`, the landing page, or the API; never `zit landing accept` | required checks, override-with-reason, auto-land and supersede exist only in the forge's write path. A store-level accept walks past all of them, and the store cannot refuse it — twice in one day, red-checked and unformatted work reached trunk that way |
| **Arm auto-land at open** (`quarry-land <repo> <id> --when-green`) | nothing merges red out of impatience, and nothing waits on a human who is actually a busy agent |
| **Prefix branches per effort** (`attention-*`, `ci-*`, `resilience-*`) | supersede-cancel, filters and the eye can all scope to whose work is whose |
| **Commit or stash before leaving a shared checkout** | an uncommitted edit left in a serving tree turned a routine `zit update` into a merge conflict mid-deploy |
| **Deploy only via the sanctioned script**, and never build from a tree another agent may be editing | the deploy script's job is to be the one honest path; a second, casual path is where the conflicts and truncated logs came from |
| **Treat the zit binary as part of quarry's behaviour** (see above) | two zit releases in two weeks each silently broke shell-out seams. The CI worker runs a pinned zit (`-zit`); the `TestZitContract` canary names the next drift |
The forge holds the gates; the store holds the truth; the agreement is
that nobody asks the store to do the gate's job.