# Working on this forge as an agent
You are an agent session with no one to ask. This guide is what you can
find out and do yourself, and what only the owner can give you. Every
route and field named here is in `GET /api/v1` and `docs/api.md`; trust
those over this page if they differ.
FORGE=https://zeitforge.app # the example host; use your own
Never put a token on a command line, in a URL or in a file you commit.
Keep it in a keychain and hand curl the header on its standard input:
printf 'Authorization: Bearer %s\n' "$TOKEN" |
curl -s -H @- "$FORGE/api/v1/me"
## 1. Find things out yourself
- `GET /api/v1` lists every route: op name, method, route, first sentence.
It also names `/api/v1/openapi.json`, which has every field's schema.
- `docs/api.md` is the same reference as prose: one section per op, input
rules, output fields.
- Is a repository name free? `GET /api/v1/repos/name?name=REPO` (add
`&namespace=TEAM` for a team). The answer has `free` and, when it is
not, `why`. To create it: `POST /api/v1/repos` with
`{"name":"REPO","create":true}`; it is then owned by the agent's person.
- Which role does an op need? The index does not say, and `docs/api.md`
shows only the floor (`token`, `admin`). The OpenAPI document does: each
op that needs a capability has an `x-quarry-caps` field, and its summary
ends "Needs capability L on the repo" ("fleet" for a forge-wide op). Letters
include `i` check in, `a` repository settings, `8` see CI runs and logs.
- A call you may not make is refused 403 with a sentence naming the
letter and who grants it: `AGENT cannot see CI runs, checks and logs on
REPO: that needs capability "8", which an admin of the repository
grants`. A repository you cannot see answers 404 as if it did not exist
(section 9).
## 2. What needs the owner, and how to ask
Send the same call again with one more header:
Quarry-Ask: <one line: why you need it>
If the call was refused 403 and your person may make it, the answer is
**202** with `{"signoff":{"id","agent","owner","op","repo","summary",
"state":"pending","requested","reason","pickup"}}`. Your person is told by inbox,
push and email and approves or denies on one page,
`/me/requests/ID`. Nothing runs until then. Approval runs exactly the call
you sent, as them, under their permissions at that moment. Read the state
with `GET /api/v1/signoffs/ID` (`request.state`: pending, approved,
denied, failed; `result` once it ran). A request waits seven days; you may
have 20 waiting.
Who may ask (`internal/api/ask.go`): an agent its person operates, or a
forge agent with an answerable admin. A repository's or team's own agent
has nobody to ask. You may ask only for what your person holds, on
repositories you can see, inside your own token's reach (one exception: a
new token or a wider reach for yourself). The person may switch asking
off for you; the call then says they take no requests from you. When the
ask is not admitted you get the plain refusal, never more.
`Quarry-Ask` goes on the call itself. A multi-step job is one request:
`POST /api/v1/requests` with `{"title","reason","steps":[...]}`.
## 3. A credential by one approval: agent.setup
`POST /api/v1/agents/setup`, sent with `Quarry-Ask`. It is never run
directly. It makes a **new** agent (a name that is or was an account is
refused), grants it, and mints its token, all or nothing.
| field | meaning |
|---|---|
| `name` | the new agent's name |
| `kind` | `push`: sync scope. `read`: api scope, reader plus CI |
| `repos` | one to ten repository names, comma separated; `repositories` is the same as a list, never send both. Duplicates and case variants collapse. One token reaches exactly these, and the role is granted on each |
| `role` | push only: `developer` (default), `publisher`, `maintainer`. Never admin |
| `days` | token life up to 366; without it the token lapses after 90 days unused |
| `belongs_to` | `repository` or `person`; empty takes the rule in RFC 0030 |
For several repositories the agent is always your own (`belongs_to` repository is refused), and if any one repository cannot be granted, nothing is made.
Do not send `ceiling`: the forge fixes it from kind and role, and a
request that sends it is refused. The owner sees the page before the
buttons: who asked and why, the new agent, its role, the credential's
scope, reach, ceiling and expiry, who collects it, and that an
uncollected credential is revoked 24 hours after approval. The agent made
stays even if the token lapses. Details: `docs/rfcs/0030-setup-by-approval.md`.
### A client with no credential
If you hold no credential at all, you cannot send `Quarry-Ask`. Start the
request yourself and let your person claim it.
1. Make a pickup key and code as in section 4.
2. `POST /api/v1/setup/start` with no authorization and the body
`{"name":"NAME","kind":"read","pickup":"CODE"}` (`kind` is `read` or
`push`). The name must be new on the forge: use `REPOSITORY-push`, or
the project's name plus `-agent`. Your person may rename it, and for
`push` may have the repository made at the claim. Send nothing else:
repositories, accounts and reasons are refused, because your person
chooses the repositories. The answer is
`{"code":"XXXX-XXXX","link":"https://FORGE/setup","expires":"..."}`.
Past five starts a minute from one address (an IPv6 /64 is one),
three unclaimed at once from it, or when the forge holds too many, it
answers 429 with `Retry-After`. It answers the same for a
name that is already an account; your person is told at the claim and
gives you another.
3. Show your person the code and the link, and nothing else of yours. They
sign in, open the link, type the code, check it matches, pick the
repositories (for `push` one, or a new private one, and the role) and
press Continue. That files an
ordinary `agent.setup` request, asked by "client of THEM", which they
approve on its page. The code lapses in ten minutes.
4. Poll `POST /api/v1/pickups/collect` with your key as in section 4. It
answers 404 until the person has claimed the code, and for a code that
lapsed or was discarded, the same as for a key that never existed; then
202 while the request waits, then 200 once. If your person asked for
your exact name and it cannot be had, it answers 410 with
`{"state":"refused"}`, and no reason: start again under another name. In the 200, `agent` is
"client of THEM" and `op` is `agent.setup`: use the name in
`result.agent` and the address in `result.remotes[0].url`, which can
differ from what you asked for.
A CI runner needs none of this by hand: `quarry-runner install
https://FORGE/r/OWNER/REPO` makes its own key, starts a request of kind
`runner` (`POST /api/v1/setup/start` with `{"kind":"runner","repo","host",
"pickup"}`, answering the short code), waits for the repository's owner to
approve it in their inbox, collects the worker token with
`Quarry-Pickup-Kind: runner`, and writes it to a private file. Show your
person the printed code and nothing else. A private repository answers as
a missing one: ask its admin for an install code and pass `--code`.
## 4. Pickup: no secret crosses between sessions
The token's value is shown once. Bind it to a key only the collecting
program holds, so the value never passes through the owner or a chat.
A program with the `quarry-runner` binary (the small download, no other
install) does steps 1 and 3 itself: `quarry-runner pickup new` writes the
key and prints the code, `quarry-runner pickup collect URL --key FILE
--clone DIR` collects it into zit. It is the same client as `quarry
pickup`; see running.md "Collect a credential". The steps below are the
wire format.
1. Make 32 random bytes. The **code** is the lowercase hex SHA-256 of the
raw bytes. The common mistake is hashing the text of the key; hash the
bytes. A code that is not 64 lowercase hex digits is refused.
umask 077
python3 - > pickup.code <<'PY'
import os, hashlib, base64
key = os.urandom(32)
open("pickup.key", "w").write(base64.urlsafe_b64encode(key).rstrip(b"=").decode())
print(hashlib.sha256(key).hexdigest())
PY
2. Ask with the code: add `Quarry-Pickup: $(cat pickup.code)` beside
`Quarry-Ask`. Without `Quarry-Ask` it is a 400. A call that leaves no
credential, or one a lapse cannot revoke (a sync token, an invite),
refuses a code. The code is public; the key is not.
3. Collect, from the program that will use it, with no query string and no
other credential:
printf 'Authorization: Pickup %s\n' "$(cat pickup.key)" |
curl -s -H @- -X POST "$FORGE/api/v1/pickups/collect" > collected.json
(An unpadded base64url key; padding is tolerated.) The answers:
- **202** `{"state":"pending","op":...}` while the owner has not
answered. Poll no faster than every 10 seconds. For an op that mints a
credential it also carries `shape`: the eventual answer's keys with
every value blank, e.g. `{"credentials":[{"agent":"","repos":[],
"scopes":[],"ceiling":"","token_id":"","expires":"","idle_lapse_days":0,
"idle":false}],"secret":{"value":""}}` (a plan: one `credentials` entry
per minting step, `secret` keyed by step number), so a client can be
tested before the one real delivery. It holds nothing of the request.
- **200** once: `request`, `op`, `agent` (who asked: you), `repo`,
`owner` (who approved), `credentials`, `secret` (for setup
`{"value":"..."}`) and
`result` (for setup: `agent` the new agent, `grants`, `token`,
`remotes` with each repository's push URL). If the call ran on your
own authority nobody approved it: `parked` is true and `answerable`
replaces `owner`.
- **410** `{"state","error"}` afterwards: `collected` (a second read
finds it gone), or `denied`, `failed`, `expired`, `lapsed`.
- **404** `{"error":"no such pickup"}`: one identical answer for an
unknown code, a wrong key and a key of the wrong shape.
- **429** with `Retry-After`: misses are counted per address and held
with a doubling wait, and a right key may call six times a minute.
Check who asked and for what in the 200 (`agent`, `op`, `repo`,
`owner`, and for setup `result.agent`) against what you asked for
before using the value: a code you showed before asking could be bound
first by someone else.
`credentials` is a list, one entry for each token the request minted
(one for setup and `agent.token.new`; one per minting step in a plan,
each with its `step`), the same for every op, so you need not know each
op's `result` shape: `agent`, `repos` (empty means every repository the
agent reaches), `scopes`, `ceiling`, `token_id`, `expires`, and for a
token with no expiry `idle: true`, `idle_lapse_days` (90), `lapses` (the
day it ends if unused from now) and `note`, a sentence saying so. It
never holds the value. `signoff.get` carries the same `credentials`
beside `secret`. Check them against what you asked for.
The 200 comes once. Keep the whole answer until the value is stored
and a push has worked; a client that reads the wrong field has lost
the credential. The value is never in `result`: it is under `secret`,
which is the last key. For setup, top-level `repo` is empty and the
repositories are in `result.grants` and `result.token.repos`, named
by key (`website`), not by address (`team/website`); the address is
in `result.remotes[].url`. A plan
(`request.plan`) answers differently: `result` is a list of steps,
each with `sentence`, `state` and `answer`, and `secret` holds each
step's secret fields, so look under `secret` for the `value`.
4. An approved credential nobody collects within 24 hours is revoked.
`GET /api/v1/signoffs/ID` never releases a pickup-bound secret.
A token with no expiry date ends after 90 days unused, not 90 days
after it was made: `credentials[].idle` is true, `lapses` gives the
day it ends if it is never used, and `note` says so in a sentence. A
token given a number of days ends on that day, used or not.
Store the value without a command line (ME is the agent you ask as, AGENT
the new one). macOS (`security -i` reads its
commands from standard input; `-w` with no value needs a terminal):
python3 -c 'import json,sys
r=json.load(sys.stdin)
assert r["agent"]=="ME" and r["op"]=="agent.setup"
assert r["result"]["agent"]=="AGENT"
print("add-generic-password -a AGENT -s forge-AGENT -w",
r["secret"]["value"])' \
< collected.json | security -i
rm collected.json
Linux: the same script, but print only `r["secret"]["value"]` and pipe it
to `secret-tool store --label="forge AGENT" service forge account AGENT`,
which reads the secret from standard input.
## 5. Push as the agent, author as a person
zit remote URL # result.remotes[].url from section 4
ZIT_PASSWORD="$TOKEN" zit push --user AGENT --once
The password is the agent's sync token, never an account password; zit
reads `--password`, else `ZIT_PASSWORD`, else a prompt. Put it in the
environment from the keychain, not on the line. `--once` uses the login for
that command alone and saves nothing into the checkout, so the agent's
login is not left there for the next session or person; without it zit
keeps a derived secret for that remote after one good push. The `--user`
on `push` is who pushes, the agent. The author of a check-in is
`zit commit --user LOGIN`, a different flag on a different command,
separate from who pushed: the forge records the agent as the pusher and each check-in keeps
the author it names.
A repository with no check-in takes the project of the first push it
receives, history and project code included. That first contact must be a
push: a pull or sync of it is refused. To start empty, `zit clone` it
before the first commit.
## 6. Tokens that fail
- A refused token (wrong, expired, revoked, wrong scope) on a route that
needs a sign-in answers **401** `authentication required`. Check with
`GET /api/v1/me/repos`.
- On a route open to anonymous callers, a refused token is not an error:
the call proceeds as anonymous. A 200 on a public repository proves
nothing about the token. A private repository answers 404.
- Every refused token is a miss at every door, against the account it
names and the address it came from: five free, then a hold of 30
seconds that doubles (a minute, two, up to about two hours). A held
account gets **429** with `Retry-After`, **even with a right token**.
A success clears the count (`LoginLimiter`, `cmd/quarry/tokendoor.go`).
On a 401, stop presenting that token. Retrying it digs the hole deeper
for you and for every other session that shares the account or address.
## 7. Never do this
Never revoke push credentials you did not make. `POST
/api/v1/me/sync-token/revoke` with neither `id` nor `repo` ends **every**
push token of the account, including other sessions'. Minting
(`POST /api/v1/me/sync-token`) adds a token and leaves the others
working (RFC 0021), and neither call can be made with a token: they need
the account itself signed in. So never ask for a person's or the forge
agent's push token. Get your own through section 3, and do not reuse
another session's credential.
## 8. CI
- A workflow is a YAML file in `ci/`, in GitHub Actions' shape
(`docs/running.md`, "CI: workflows, runners and secrets"):
on: [push, landing] # a check-in on a branch; a landing's merge
jobs:
test:
timeout-minutes: 30
steps:
- run: make test
Keys: `on` (`push` and `landing` with `branches` or `branches-ignore`,
`tag`, `workflow_dispatch`, `schedule`), `jobs`, `needs`, `environment`,
`timeout-minutes`, `env`, `steps`, `name`, `run`, `working-directory`.
There is no `if:`, no expression but `${{ secrets.NAME }}`, and any
other Actions key (`uses:`, `runs-on:`, `strategy:`, `container:`) is
refused by name: the runner checks out for you, and tools must be on
its host. Check a file before pushing it:
`POST /api/v1/r/REPO/ci/pipeline` with `{"path","content","validate":true}`
(needs check-in rights); no `diagnostics` means it is accepted.
- A run's `status` is its verdict: `completed` means every job passed,
`failed` that one did not, `running` that it has not ended.
- A secret reaches a job only through an environment it names, on a
branch that environment admits (protected branches unless it lists
others), and never in a landing's merge run.
- A job log opens with a block between `::quarry-job-environment::` and
`::end-quarry-job-environment::`: user, umask, working directory,
`PATH`, variable names (never values) and tool versions (`absent` if
missing). Read it first when a step behaves differently than locally.
- A workspace is reused between runs: the checkout is updated in place
and untracked files a job wrote stay, `HOME` included. Treat that as a
cache, never as something a test may rely on.
- `GET /api/v1/r/REPO/ci/runs/RUNID` and `.../ci/runs/RUNID/log` (query
`step`, `tail`); `GET /api/v1/r/REPO/ci` lists runs.
- Both need capability `8`. Without it: 403 `cannot see CI runs, checks
and logs on REPO: that needs capability "8"`. A reader role holds it, an anonymous caller never does.
- Which checks a landing onto a branch must pass: `GET /api/v1/r/REPO/ci/required-checks`
lists the rules (capability `a`); `POST` the same path with
`{"branch","checks":[...]}` sets one (`branch` a name or glob, an empty
`checks` removes the rule). An agent is refused the POST: send it with
`Quarry-Ask` and your person decides.
- Do not poll. `GET /api/v1/r/REPO/events/stream` (all repositories you
read: `/api/v1/events/stream`) is Server-Sent Events. Filter with
`kind`, exact or a pattern, repeatable: `?kind=ci.completed&kind=landing.*`.
A `ci.completed` carries `runId`, `checkin`, `pipeline`, `status`. Resume
with `Last-Event-ID`.
- A job's environment (`internal/runner`). Each step is its own
`sh -e -c`, in the checkout (or the step's `working-directory`).
Without `--job-users` the job runs as the runner's own user with the
runner's whole environment plus `CI=true`, gets no run token and may not
use secrets. With job users it runs as one of the pool's users through
sudo with umask 007 and only: `PATH` (the runner's), `HOME` (the
workspace), `TMPDIR` (`.quarry-tmp` in it), `CI=true`, `LANG` if set,
`QUARRY_USER` and `QUARRY_TOKEN` (this run's token), `QUARRY_TAG`,
`QUARRY_CHANGED`, `QUARRY_TOUCHED`, the job's and step's `env`, and the
secrets it names. Which compilers and tools exist depends on the
runner's host. Print it from a step:
`id; umask; env | sort; command -v cc fossil zit`.
## 9. A private repository, from outside
It is answered as a missing one, so its existence is not told. The page
and the API answer 404 `no repository "REPO"`. Sync is answered as a
Fossil server answers it, the same for a missing repository: a caller
signed in as nobody is told `not authorized to clone` (or to read), and a
login that does not check out `login failed`. If you get either on a
repository you were told exists, you may lack a role on it: ask your
person (section 2), do not retry.
Name the account on every sync, the first clone included. Tested forms:
ZIT_PASSWORD=TOKEN zit clone --user ACCOUNT URL
ZIT_PASSWORD=TOKEN zit push --user ACCOUNT
fossil clone --save-http-password https://ACCOUNT:TOKEN@HOST/r/REPO REPO.fossil
fossil pull; fossil push; fossil sync # bare, from then on
A zit older than 0.14.8 wants the name in `--user`: with the name inside
the URL it sends an empty password unasked and the answer is 404. From
0.14.8 `https://ACCOUNT@HOST/r/REPO` works too. Never put the token in a
URL for zit. fossil (2.28) needs `--save-http-password`: with no terminal
it otherwise drops the password after the clone, and refuses every later
command itself, before anything is sent. With it, bare `fossil pull`,
`push`, `sync` and the autosync on commit work. Do not put the
account in the URL without its token. This is the one place a token goes
on a command line; read it into a shell variable from your secret store
and never write that URL to a file.
The password is a live sync token or push credential of that account,
never its account password or an API token. Every refused credential
counts as a failed sign-in: five in a row put a hold on the account that
doubles each time, and a correct login is refused while it lasts. So on a
refusal, check the name and the token, then ask your person about the
role; do not probe with variations.