Timeline

Check-in 2cacce9b01aa added obs.Default.Spawned, RecordSubprocess, RecordRender and Snapshot calls across zitcli, web and api, but the package itself was never added — so trunk did not compile at all.

The files are the ones from the authoring session's tree, unmodified, tests included. Nothing here is my design; this only makes the import resolve so the tree builds again. check-in: 5229b05522 user: dmestas tags: trunk

The panels were steady-state numbers and said nothing while a fleet came up — which is the window an operator most wants to watch and the one where a forge most looks hung. A multi-repo re-ingest on this fleet was mistaken for a hang precisely because nothing said '23 of 25 stores ready' while it worked.

A phase banner leads the page: starting, indexing, syncing or rest, with stores ready over stores served, artifacts this session, the slowest recent round, and the names of what is rebuilding — which, not merely that something is.

Derived rather than tracked. Every store already reports Indexing, LastOK, LastCount, Total and Stalled; a phase field somewhere would be one more thing to keep true. The test ordering is the reader's priority: rebuilding outranks syncing, and a store that has never finished a round outranks both.

A stalled store is DONE waiting — it needs a person, not time — so it does not hold the fleet in 'starting' forever and does not sit in the bar as pending. Ready and stalled are drawn as two segments of one track, or the bar could never fill. closed check-in: 78fbef9d30 user: dmestas tags: ops-phase

quarry could not answer two questions about its own behaviour. A render time was printed in the page footer and discarded, so a 4.24s files page was visible only to whoever loaded it and read the footer. Subprocess load was visible nowhere: a warm loop once ran 134 concurrent `zit ls --long` against the same hash, load average 546 on 14 cores, and /fleet said "all healthy" throughout.

/fleet/ops, site-admin only. Slowest routes by p95 — a mean hides the tail, and the tail is the thing worth finding. Store subprocesses rolled up per verb, with the number that actually mattered: how many run at once, and the peak, which outlives the incident. Recent renders with ops time and the zit time inside it.

Instrumented at the two seams that cover everything: the ops registry, which every page's data crosses, and the outermost middleware, which is the only place the template's own cost is known — the footer's breakdown is assembled before the template runs, so a page can never report the time it spent rendering itself.

Datastar with no no-script path, deliberately: an internal dashboard that cannot move while an operator watches a fleet index is worth much less than one that can, and no consumer depends on it. The op behind it is registered and versioned like any other.

The rings are small and in memory. This is the snapshot; history belongs in the analytics store behind obs.Sink, which is an interface with a no-op default so the persistent implementation can be cgo without the request path importing it. closed check-in: 022966bc05 user: dmestas tags: fleet-ops

zit ls --long adds size, permission and text/binary, and the last of those reads every listed blob. On a large store the annotated forms cost ~18ms per file where the plain two-field listing costs ~0.09ms: measured on tcl, 0.20s plain against 38s for --size and 41s for --long. The cost scales with the STORE, not the listing (filed as zit#148).

Four callers read only .Name and threw the rest away: the About marker scan, a root check-in's file set, the doc index, and CI's file list. They use LsNames now, which runs the plain listing and caches it beside the annotated one.

The tree page and its warm still need the columns they display, so they are unchanged; that cost is the upstream issue's to fix. check-in: 78db482057 user: dmestas tags: trunk