Skip to content
quarry Artifact [ce29e11c64]
Login

Artifact [ce29e11c64]

Check-ins Using Download Text

Artifact ce29e11c64e7a64cd054bd90891a1c49f477ab318a8bc95dda880f6bfbb2d1dd:

Summary

quarry names the things that happen in a repository, once, in a vocabulary that pipelines, webhooks and notifications all subscribe to. An event is a noun and an action — ticket.closed, landing.opened, branch.created — carried on the bus that already exists, with a payload shape declared per noun.

A pipeline runs from one of three sources, and only the first is an event: something happened, the clock said so, or a person asked (§2.3). Schedules and ad-hoc dispatch are repository configuration gated on fossil's a capability, so a repository's own admin owns them (§2.4).

Today there are two and a half paths and no vocabulary. Webhooks derive an event's kind by slicing a notification's idempotency string. CI does not use events at all: it polls the index for new check-ins behind a per-repository watermark. Notifications have their own workflow names. Adding one more trigger — a landing opening, say — means either a third path or generalising anyway, so this RFC generalises first.

The vocabulary is the load-bearing part, and the reason this is an RFC rather than a commit: a pipeline definition names events in its on: clause, so an event name is a published interface. Renaming one breaks every pipeline in every repository that used it, in a system where the pipeline definition is itself a versioned artifact in someone else's store.

Motivation

What exists

ci.Trigger runs after every fold. It asks the index for check-ins newer than a watermark, reads ci/*.json at each one, registers the definition at that check-in, and starts a run. pollCI later polls unfinished runs and, on a terminal status, mints a ci-pass / ci-fail tag on the check-in.

That is a complete, working pipeline trigger for exactly one event, with the event's identity left implicit. Four things in it are check-in-shaped:

today needed
trigger poll NewCheckins(watermark) subscribe to a named event
payload {repo, checkin, branch, date, user, comment, cloneUrl, pipeline} per-noun, common envelope
watermark one check-in per repository per subscription
verdict sink WriteTag(verdict, checkin, runID) per-noun; a landing wants a reply

Every one of those blocks the landing trigger (quarry ticket 49323c33), and none of them is made easier by doing the landing trigger first.

What the field does

Four vocabularies were read before proposing a fifth.

Fossil hooks (www/hooks.md) are artifact-level and carry no semantics at all. Three types — after-receive, before-commit, commit-msg — and the after-receive hook's stdin is a list of artifact hashes with human-readable descriptions. There is no "on branch created" or "on merge". The doc states its purpose plainly: to "start up a background testing or CI process".

Fossil alerts (src/alerts.c) carry fossil's only semantic vocabulary, as subscription letters on the subscriber.ssub column:

a Announcements          c Check-ins            f Forum posts
n New forum threads      r Replies to my posts  t Ticket changes
u User permission changes  w Wiki changes       x Edits to forum posts

Eight types. It splits the forum four ways by verb, and has no event for a branch, a tag, a technote or a merge.

GitHub publishes 75 webhook events, most of which are not about version control: roughly six VCS core, fifteen collaboration, eight CI, ten security scanning, fifteen governance, and twenty platform and commerce events (marketplace, sponsorships, apps, Pages, Projects). quarry has a concept for about twenty of them. Gitea publishes about 27, GitLab 15.

The shape decision

The three forges disagree about the axis, and the disagreement is the design question:

quarry takes GitHub's. Under Gitea's rule every new verb is a new permanent event name, and the name is a published interface. Under GitHub's, a subscriber can ask for ticket and get every change, or ticket.closed and get one, and adding a verb later costs nothing. Fossil's split shows the cost of the other choice inside one product: f and n and x are three names for three verbs on one noun, and a subscriber who wants "any forum activity" must name all three and remember to add the fourth when it appears.

Guide-level explanation

An event is noun.action. A subscriber names either the whole noun or one action of it:

"on": ["checkin.created"]                  one action
"on": ["ticket"]                           every action on the noun
"on": ["landing.opened", "landing.reopened"]

A pipeline with no on: clause runs on checkin.created, which is today's behaviour exactly, so every pipeline already in a repository keeps working without being edited.

Every event carries the same envelope, and a payload keyed by its noun:

{
  "event":  "landing.opened",
  "repo":   "quarry",
  "at":     "2026-08-24T09:48:57.123",
  "actor":  "dmestas",
  "id":     "activity:landing:9802c157…",
  "payload": { "landing": "9802c157…", "branch": "wiki-struct", "target": "trunk" }
}

id is the idempotency key. It is stable for one occurrence of one event, and it is how a consumer that sees an event twice can tell that it has.

Reference-level explanation

§1 The vocabulary

Nouns and their actions. An action marked † is not yet raised by any code path and is reserved so that adding it later is not a vocabulary change.

noun actions payload
checkin created hash, branch, parents, user, comment
branch created, moved, closed† name, tip, previous tip
tag created, removed† name, target, value
ticket opened, changed, closed, reopened id, title, fields changed
wiki edited, created, deleted title, hash
technote created, edited event id, hash
forum posted, replied, edited†, moderated thread root, post hash
landing opened, commented, accepted, rejected id, branch, target
ci started, completed run id, checkin or landing, verdict
mirror pulled, failed url, artifacts received
repo created, removed, settings name
clone cloned, fetched actor or anonymous, artifacts sent
account added, caps, removed account, repository, caps

Fourteen nouns, thirty-odd actions. For comparison: fossil's alerts name eight things, GitHub's 75 events name about twenty that quarry has a concept for.

Clone counts. clone.* is observable because every sync request goes through one place: Fronter proxies POST /r/{repo}/xfer and already runs a ReadGate per request, so the choke point exists. Counting fetches is free at that point. Distinguishing a clone — a first fetch by a peer holding nothing — from an incremental pull requires reading the request body, since that distinction lives in the sync protocol's announcement and nowhere else. Both counts are operational per §2.1 and so are recorded in OLTP as they happen; the bus carries notice of a row that already exists.

GitHub reports clones and unique cloners separately. Uniqueness needs an identity to count by, and a sync request may be anonymous, so quarry counts unique by account where there is one and reports anonymous clones as a separate figure rather than guessing at a person behind an address.

Three of these exist in no other forge, and are here because quarry has the concept and an operator needs the hook: technote (fossil has the artifact but no alert for it), mirror (a forge that tracks 25 upstreams needs to fire on a failed pull), and landing.

§2 What the bus is, verified

Read from internal/events/events.go rather than assumed, because §2's promises depend on it:

desired := &nats.StreamConfig{
    Retention: nats.LimitsPolicy,
    Discard:   nats.DiscardOld,
    MaxBytes:  1 << 30,   // 1 GiB ceiling across all repos
    MaxAge:    time.Hour, // live window; older needs a reindex
}

The stream is a one-hour live window, not a log of record. It is a transport between the store and the index, and the durable truth is the pair either side of it: the store (authority) and the publisher cursor in the KV bucket, which advances only after a fold succeeds. A consumer that falls outside the window does not catch up by replaying; it rebuilds from the store.

Two consequences this RFC must respect rather than wish away:

  1. A consumer down for an hour has missed events permanently. CI's current watermark-and-poll survives that, because it reads the index, which is durable. A naive move to bus subscriptions would trade a robust trigger for a fragile one — §2.1 is how that is avoided.
  2. Per-node durable consumers each read the whole stream (limits retention was chosen over work-queue for exactly this: every node must fold every event). So three consumers subscribing independently is the shape the bus was already built for.

§2.1 Two classes of event, and why the distinction is load-bearing

The one-hour window is survivable for some events and not others, and the line is whether the event is derivable from the DAG:

Derived events — checkin, branch, tag, ticket, wiki, technote, forum, landing. Every one is a reading of artifacts the store holds. A consumer that missed one can re-derive it by walking the index, exactly as the ingest pipeline rebuilds after a gap. A missed event costs latency, not truth.

Operational events — ci, mirror, repo, account, clone. None of these is in the DAG. mirror.failed is a fact about a network attempt; account.caps is OLTP state; a clone is a request that happened. If the delivery is missed, the fact is gone.

So operational events MUST be recorded in OLTP as they are raised, and the bus carries a notification of a record that already exists. Derived events need no such backing — the store is their backing.

This is not symmetry for its own sake: it is the difference between a mirror.failed an operator can page on and one that silently did not happen.

§2.2 Delivery semantics, which the medium decides

Fossil's hook documentation states the constraint, and quarry inherits it rather than choosing it:

A push might not deliver all of the artifacts for a checkin. If Fossil knows that a /xfer HTTP request is incomplete, it will defer running the after-receive push for 60 seconds, or until a complete /xfer request is received. This helps to prevent after-receive hooks from running when incomplete checkins exist in the repository, but it does not provide hard guarantees, as there is no way to do that in a distributed system.

So:

  1. At-least-once. A consumer may see an event twice. id is the idempotency key and a consumer that acts on an event MUST be idempotent under it. quarry's own notification plane already works this way.
  2. Possibly batched. Fossil warns that two pushes' artifacts "might be combined into a single after-receive callback". quarry's fold has the same property; a single fold can produce many events, and a consumer must not assume one event per delivery.
  3. Possibly partial. An event may describe state that is still arriving. A consumer that needs the whole picture reads the index at handling time rather than trusting the payload to be complete.
  4. Ordered within a repository, not across. The bus preserves order per repository because the fold does; nothing orders two repositories against each other, and nothing should be written that needs it.

§2.3 Three trigger sources, of which events are one

A pipeline runs because something happened, because the clock said so, or because a person asked. Only the first is an event, and conflating the three is how "on:" ends up meaning two different things.

source on: what it carries
event checkin.created, landing.opened the event envelope of §1
schedule schedule: "0 * * * *" the fire time and the ref it runs against
dispatch dispatch: true the actor and any inputs they supplied

This is GitHub's split (schedule, workflow_dispatch) and it is the right one: a cron run has no actor and no artifact, and a dispatched run has an actor but nothing happened. Forcing either into the event envelope would mean a payload full of fields that are empty by construction, which is how a schema stops meaning anything.

Schedules and dispatches are OLTP state, not artifacts. A schedule is a forge configuration — which repository, which pipeline, which cron expression, enabled or not — and it is exactly the shape of the existing webhook table. Both are per-repository, and both survive a reindex because neither is derived from the DAG.

Ad-hoc runs need no new engine. dagnats already registers a workflow from a definition and starts it with an input map; that is what ci.launch does today. A dispatch is the same call with a different input, and a schedule is the same call on a timer. What is missing is not execution but the record of what should run and when, and the surface to manage it.

§2.4 Who may schedule and who may dispatch

Both are repository configuration and both spend the engine's time, so both follow RFC 0004's rule for configuration: they take a — fossil's Admin capability, the same letter that already gates repo.update, repo.reindex, ticket.config(write) and the mirror verbs.

That answers "repo admins should have these" in fossil's own vocabulary rather than by inventing a role: an account holding a on a repository can create, edit, disable and delete its schedules, and can dispatch a run. An account without it can see that a schedule exists — a pipeline that runs nightly is not a secret from the people reading the repository — and cannot change it.

Deliberately NOT a new capability letter. RFC 0004 §6's rule is that a letter which exists in fossil carries its fossil meaning and a surface which exists only in quarry names an existing letter rather than minting one; a forge that grows private letters stops being a fossil forge by increments.

One consequence worth stating: a excludes s, so a repository admin may schedule work on their own repository and not on anyone else's, and a fleet-wide schedule is a site-admin action. That falls out of the cap model rather than needing its own rule.

§3 Handlers do not run inside the fold

Fossil again:

Fossil holds a write transaction on the repository while the after-receive hook is running.

A hook that writes to the repository it was fired by therefore deadlocks. The same is true here for a different reason: quarry mints artifacts through the lease holder, and the fold is not it.

An event handler that mints an artifact runs outside the fold, through the lease holder, asynchronously. The CI verdict already obeys this by accident of structure — pollCI runs on a timer, not in the ingest path. This RFC makes it a rule rather than an accident, because the landing verdict (700b22b6) would otherwise be written from exactly the wrong place.

§4 One stream, three consumers

consumer today after
webhooks Raise() from fireNotify, kind sliced out of a TxnID subscribes by name
CI polls NewCheckins behind a watermark subscribes by name
notifications own workflow names subscribes by name

The watermark does not disappear: it becomes per-subscription rather than per-repository, because two consumers may be at different positions and one falling behind must not advance the other.

§5 Verdict sinks

A CI run's verdict is an artifact, because 0001 §6's no-hostage rule says the result must clone with the repository. Which artifact depends on the noun that triggered it:

The rule generalises: the verdict takes the shape of the thing it judges.

§6 What is deliberately absent

Named so the absence reads as a decision rather than an oversight:

The last one is the one most likely to be asked for later. It should be its own RFC, and it should not reuse these names.

Drawbacks

A published vocabulary is hard to change. That is the point, and also the cost: a name chosen badly here is one every pipeline in every store repeats. The mitigation is the reserved actions in §1 and the noun-plus-action shape, which absorbs new verbs without new names.

One stream is a single point of failure where today CI's polling is independently robust: if the bus stops, CI stops. The watermark survives, so nothing is lost, but a stalled bus becomes a stalled forge in a way a stalled poll loop is not. §2's at-least-once rule and per-subscription watermarks are what make recovery a replay rather than a repair.

Schedules and dispatches put quarry in the business of running things nobody asked for at that moment. A cron pipeline consumes engine time on a repository whose owner may have forgotten it exists, and unlike an event-driven run there is no push to point at. The a requirement in §2.4 bounds who can create one; nothing here bounds how many, and a per-repository ceiling is probably wanted before this ships.

Fourteen nouns is more than fossil has and fewer than GitHub. Both comparisons will be made, and neither is the argument: the set is what quarry can honestly raise today plus what it will raise next.

Rationale and alternatives

Do the landing trigger without a vocabulary. Rejected: all four generalisations in the motivation table are needed for it anyway, and the result is a third trigger path. Ticket 49323c33 says so in its own words — "rides the existing event plane … rather than growing a second trigger path".

Adopt fossil's after-receive shape: hand the handler a list of artifacts. Rejected. It is honest and it is what fossil does, but it pushes the whole interpretation problem to every consumer: each pipeline would re-derive "is this a new branch" from artifact hashes, and they would derive it differently. quarry already folds artifacts into meaning; the event stream should carry the meaning it already has.

Adopt Gitea's noun_verb naming. Rejected in §Motivation: every new verb becomes a permanent name.

Use GitHub's names verbatim where they overlap. Tempting for recognisability, and rejected in two places: issues is called ticket here because that is what fossil calls it and fossil compatibility is the north star, and pull_request is called landing because it is not one — nothing is transported and nobody requests a pull (zit#127).

Prior art

Unresolved questions

  1. Does branch.moved fire per check-in or per fold? A ten-commit push moves the branch once but creates ten check-ins. Firing per fold makes branch.moved cheap and coarse; firing per check-in makes it redundant with checkin.created. Leaning per fold, with the check-in count in the payload.
  2. Should ci.completed be subscribable? It would let a pipeline trigger on another pipeline, which is either composition or a foot-gun. GitHub allows it (workflow_run) and guards against recursion. Deferred until something needs it.
  3. How does a subscription express "my repository only" versus fleet-wide? The webhook plane is per-repository today; CI is per-repository; a fleet-wide subscriber (an operator watching mirror.failed across 25 stores) has no way to say so.
  4. ~~JetStream delivery guarantees have not been traced.~~ Resolved and folded into §2: the stream is a one-hour window with limits retention and per-node durable consumers, and the durable truth is the store plus the publisher cursor. The consequence — that operational events need OLTP backing and derived ones do not — is §2.1.

Future possibilities