Timeline

A call on context.Background() carries no timer, so a page blocking six seconds on a warm pass reported 'ops 11.09s, zit 32.9ms' — and the ops panel's claim that ops minus zit is index work was false while it did. Overlap catches what attribution misses: subprocess busy time inside the render window, unioned, reported only when it exceeds what was already attributed. check-in: 17454ef71f user: quarry tags: trunk

Recent renders showed the route pattern, so a dozen rows all read GET /r/{repo}/timeline and none could say which repository was slow. The URL is now recorded and shown, with the route beside it as the link to its drilldown.

Two bugs found by clicking through rather than by testing the parts:

queryesc double-encoded every URL it touched, because html/template already escapes a value in query context. The routes panel linked to its own drilldown with %252F for a slash, so every link 500'd. Six call sites, all wrong; the function is gone.

The days control I added with the trend forwarded an empty parameter, and a numeric field with a default takes it when the key is MISSING and refuses when it is present and empty.

Routes panel gains a filter and a sort; renders gain a deeper-history page reading the store rather than the ring. closed check-in: 2d033f106e user: dmestas tags: ops-paths

The store recorded renders and nothing read them. The panels are one process's memory, so the page could say how fast a route is now and never whether it is worse than it was — which is the question after a deploy.

The drilldown gains a per-day table from the store: p50, p95, max. Nearest-rank percentiles, matching how the snapshot computes them, so two numbers on one page both called p95 agree.

Empty for no traffic and empty for history-off are different answers and the page gives the right one. closed check-in: a0fa952495 user: dmestas tags: landing-merge

The live panels read in-memory rings, so a forge whose store failed to open looked exactly like a healthy one. Sink.Err() existed and nothing asked it. The page now distinguishes three states: off, broken, and recording to a named path.

Retention was 'append for ever', which is a default nobody was asked about. --analytics-keep sets the window, 14d by default, 0 keeps everything. Pruned at startup and daily. check-in: e2b74749b4 user: dmestas tags: fleet-census

The panel finds a slow route; this says why. Two things the roll-up cannot show.

WHICH REPOSITORY. A route pattern folds pikchr and tcl into one line, and repository size is the dominant cost on this fleet — GET /r/{repo} at 420ms could be either, and the two differ by orders of magnitude.

WHICH CALLS. Each recent request is listed with the store calls it made, in the order it made them, and what they cost together. A request with no calls listed spent its time somewhere else, which is itself the answer when the store is not to blame.

Reads the same rings the panels read, so a request that has aged out simply is not there. That is the honest limit of a snapshot: this answers what a slow request just did, not what happened yesterday. check-in: bb6ec6b896 user: dmestas tags: ops-drill

The panel could find a slow route and never say why: the render list and the subprocess list were two independent tables, so 'GET /search took 1132ms' and 'zit artifact ran 499 times' could not be connected even when one caused the other.

Every request is now tagged at the outermost middleware, and every store call records the tag. Work with no request behind it — ingest rounds, warm passes, the doc indexer — records the empty string, which is itself the useful distinction: store load with nothing to blame is background, and telling the two apart is most of what the subprocess panel is for.

Also fixes a silent hole of the same shape as the route-pattern bug: Repo was read with req.PathValue in the outermost middleware, where the router has not set it — the values live on the request the mux passes DOWN. Every render recorded until now carried an empty repo, 61 of 61 rows in the analytics store, which made the numbers unattributable on a fleet where repository size is the dominant cost. Read from the path instead. check-in: bc35cc0c45 user: dmestas tags: ops-drill

Same mistake as before: extras was printed and then the commit ran without the add. Caught by checking the check-in rather than the worktree, which is the only check that works. closed check-in: 63ad319ef2 user: dmestas tags: ops-budget

500ms was a literal in a template and a constant nowhere. The panel coloured a p95 over it and nothing else knew, so the forge could show a route in red and tell no one.

obs.Budget is now the single definition. The panel colours against it — sent on the wire rather than hardcoded, so the page and the threshold cannot disagree — and crossing it triggers a notification.

First crossing only, on errlog's rule and for errlog's reason: the same page is slow on every request until somebody fixes it, and an alert channel that repeats itself gets muted. A muted channel is worse than none, because it reads as healthy. The decision is made under the lock, so two slow requests racing on one route cannot both report it.

This is the difference between an operator watching the forge and the forge watching itself: the ops page has to be looked at; a notification arrives. check-in: fc8ef2165b user: dmestas tags: ops-budget

Check-in 2cacce9b01aa added obs.Default.Spawned, RecordSubprocess, RecordRender and Snapshot calls across zitcli, web and api, but the package itself was never added — so trunk did not compile at all.

The files are the ones from the authoring session's tree, unmodified, tests included. Nothing here is my design; this only makes the import resolve so the tree builds again. check-in: 5229b05522 user: dmestas tags: trunk