Check-in [96b30ad4f5]

500ms was a literal in a template and a constant nowhere. The panel coloured a p95 over it and nothing else knew, so the forge could show a route in red and tell no one.

obs.Budget is now the single definition. The panel colours against it — sent on the wire rather than hardcoded, so the page and the threshold cannot disagree — and crossing it triggers a notification.

First crossing only, on errlog's rule and for errlog's reason: the same page is slow on every request until somebody fixes it, and an alert channel that repeats itself gets muted. A muted channel is worse than none, because it reads as healthy. The decision is made under the lock, so two slow requests racing on one route cannot both report it.

This is the difference between an operator watching the forge and the forge watching itself: the ops page has to be looked at; a notification arrives. check-in: fc8ef2165b user: dmestas tags: ops-budget

The rings die with the process, and this fleet restarts constantly — a 4.24s render and a 546 load average both left no trace. obs.Sink had one implementation and it did nothing; this is the durable one.

DuckDB because the workload is columnar: append-only, time-ordered, aggregate-queried, never point-looked-up by a page. RFC 0001 defers the lake to 'when hydration and analytics — the things columnar is for — exist', and reserves the operational surface for SQLite. This is that analytics workload, not a second operational store. Wide flat tables, no keys and no indexes, which is also what a Parquet COPY TO wants when the lake arrives.

Isolated behind a build constraint, in its own package, imported only by the composition root. Both halves of the reason were measured rather than assumed: CGO_ENABLED=0 fails on go-duckdb's bindings, and GOOS=js reports 'build constraints exclude all Go files'. Leaving the import off the request path is not enough — any reachable import breaks a wasm build — so cmd/quarry has two files, one per side of the tag. Verified: GOOS=js GOARCH=wasm builds cmd/quarry clean at 76MB, without DuckDB.

Buffered, never a write per request, and flushed on a timer so a quiet forge still lands rows rather than holding them in memory for days.

Failure is not fatal. The first run failed on 'at' being reserved in DuckDB; the server started anyway, the panels worked, and the log said what was wrong. Analytics is derived data about a forge that works without it.

Costs, measured: +462MB module cache, binary 57MB -> 104MB, cold build +10s, and CGO_ENABLED=0 no longer builds the server. closed check-in: 564f272d3c user: dmestas tags: ops-history

Seven test files were written this session, reported as landed, and never committed: `zit commit` with no paths captures TRACKED files, and new files need `zit add` first. make check passed every time because they were on disk in the worktree, which proves nothing about what is in the check-in.

What was missing:

- search kinds, held in agreement across four places (the index list, two ops validators, the scope chips, hitOf). This is the guard that caught `doc` and again `file` the moment each was added; without it the validator silently refused a kind the chips offered. - technote edits collapsing to one timeline row, folded in three different orders, because a clone delivers artifacts in wire order. - wiki and technote search matching the CURRENT revision only, the "terrified" case. - Contributors(0) meaning every contributor rather than a silent 100. - the ops phase states, the stalled-store trap, and an empty fleet.

Verified by removing them and running the suite: it stays green, which is precisely the problem — trunk has been passing on their absence. closed check-in: 24fdaa723a user: dmestas tags: search-guards

Check-in 2cacce9b01aa added obs.Default.Spawned, RecordSubprocess, RecordRender and Snapshot calls across zitcli, web and api, but the package itself was never added — so trunk did not compile at all.

The files are the ones from the authoring session's tree, unmodified, tests included. Nothing here is my design; this only makes the import resolve so the tree builds again. check-in: 5229b05522 user: dmestas tags: trunk

The panels were steady-state numbers and said nothing while a fleet came up — which is the window an operator most wants to watch and the one where a forge most looks hung. A multi-repo re-ingest on this fleet was mistaken for a hang precisely because nothing said '23 of 25 stores ready' while it worked.

A phase banner leads the page: starting, indexing, syncing or rest, with stores ready over stores served, artifacts this session, the slowest recent round, and the names of what is rebuilding — which, not merely that something is.

Derived rather than tracked. Every store already reports Indexing, LastOK, LastCount, Total and Stalled; a phase field somewhere would be one more thing to keep true. The test ordering is the reader's priority: rebuilding outranks syncing, and a store that has never finished a round outranks both.

A stalled store is DONE waiting — it needs a person, not time — so it does not hold the fleet in 'starting' forever and does not sit in the bar as pending. Ready and stalled are drawn as two segments of one track, or the bar could never fill. closed check-in: 78fbef9d30 user: dmestas tags: ops-phase