Skip to content
quarry Documentation
Login

Documentation

Summary

Every forge has an activity page and every one of them answers the wrong question. GitHub's contribution graph shows how much; Fossil's /reports shows how many, by bucket. Neither shows who was working on what, when, which is the thing a person opening an unfamiliar repository actually wants to know, and the thing a maintainer wants to see about their own.

This RFC adds two views drawn from one data source:

One rule governs both and is the most important sentence in this document: a merge check-in's file list is the absorbed branch's work, not the merger's. On fossil-family stores merges are ~10% of check-ins but carry ~60% of all file touches; counted naively they inflate some people's footprint by 2–20×. Quarry counts merge-carried files as inherited and excludes them from every ranking. Measured numbers are in §3.

Both views are one ops-layer query rendered two ways: server-side SVG that works with JavaScript off, and a canvas that adds hover, drag-zoom and the tree axis at scale. No new ingest, no new plane, two small index changes.

Motivation

What the other forges offer, and why it is not enough:

forge page shows does not show
GitHub contribution graph; Insights › Contributors / Commits / Punch card volume per day; per-person commit counts over time; hour×weekday heatmap who, on the same chart as when; what was touched; anything per-file
GitLab Analyze › Contributors, Repository analytics per-person line charts; commits by day/hour who and what; anything on one screen
Gitea / Forgejo Activity, Contributors period summaries, counts, a per-person chart when in the day; what was touched
Fossil /reports counts by user, file, year, month, week, weekday, hour any two dimensions together; anyone but the top of the table

All of them separate who from when from what, and all of them count merges as authorship. The prototype put the three on one plot and showed things the tables hide: that one contributor's "over-representation" was merge inheritance, that their real work was documentation (41% of authored check-ins), that a project's working hours shifted by eight when its lead moved, and where a release landed relative to the burst that produced it. People looked at it and asked questions about the repository, which is the whole test.

The repository home page gets the core slice for the same reason GitHub's home page gets the green graph: it is the first thing a visitor's eye lands on, and it should say something true. "Green squares" says busy. The slice says drh, with dan since 2016, released every few months, here are the gaps.

Guide-level explanation

A visitor opens /r/sqlite. Under the tree and the counts there is a core slice: the last year as a row of small coloured cells, one per day, with a legend of the people whose colour they carry. Clicking the slice opens /r/sqlite/activity.

The activity page shows the core sample: the last ~900 days of check-ins as dots, hour of day on Y, one colour per person, marginal bars along the top (per day) and right (per hour). Below it, every year of the slice. Two controls: clock / tree, and a branch select. On the tree axis, Y becomes the path, sorted so directories band together, and a check-in becomes one dot per authored file. Hovering names the check-in, the person, the file; dragging zooms the window. With JavaScript off, the same picture is a static SVG with <title> hovers and the controls are plain links.

A person page (/r/{repo}/people/{name}) shows the same core sample with everyone else ghosted. There is no person control on the activity page; the person page is the person filter, and the legend is the door.

A maintainer reading it sees merges as check-ins on the clock axis (they happened) but not as files on the tree axis (they were not written here). The count beside their name is authored check-ins.

A client calls GET /api/v1/r/{repo}/activity and gets the rows the chart is drawn from, and GET /api/v1/r/{repo}/activity/days for the per-day aggregate behind the slice. docs/api.md describes both.

Reference-level explanation

1. What the index already has

The fold (internal/index/index.go) keeps, per check-in:

That is the whole data model. Nothing is re-ingested.

2. Two index changes

Both ALTER TABLE ADD COLUMN / CREATE INDEX in the schema string, filled by the next rebuild; a repository indexed before them reports directories as files until then, which is ugly, not wrong.

  1. filechange.dir INTEGER NOT NULL DEFAULT 0 — set to 1 for the ancestor rows foldChangePaths synthesises. The tree axis and every file count want leaves only; without the flag, leaf detection is a self-join per row.
  2. CREATE INDEX filechange_by_checkin ON filechange (checkin) — the activity query walks check-ins by date and fans out to their files; the existing (path, date) index cannot serve that direction.

3. The merge attribution rule

Measured with zit crosslink over three stores (the prototype's source; the fold sees the same file lists):

store check-ins merges touches carried by merges
fossil 20,937 11% 61%
sqlite 36,853 10% 58%
libfossil 3,879 — —

Per person, touches counted naively ÷ touches authored (fossil): ashish 20.15×, ashepilko 4.36×, george 3.70×, jan.nijtmans 2.34×, brickviking 1.76×, drh 0.55× (drh's own work is absorbed by others' merges). The naive count is not a small error; it is a different chart.

Rule. A check-in with a non-primary parent (checkin_parent.ord > 0) is a merge. Its filechange rows are inherited: they count for "this check-in touched N files" in a tooltip, and nowhere else. They are not dots on the tree axis, not bars in a marginal, not files in a person's total, not the lead-author signal in a slice cell. The merge itself remains a check-in — it appears on the clock axis and in every check-in count — because it happened.

Known imprecision, accepted. A merge that also carries hand edits (conflict resolution, a fix-up in the same commit) loses those edits from the tree axis. Recovering them means diffing the merge's file set against the absorbed branch's file set since the fork, which the index can do later (§Future) and which the prototype showed is not needed for the picture to be right.

The rule lives in the op (§4), not in the renderer. Both views, the API, and anything built on it later see the same numbers.

4. Ops

Two queries, registered in internal/api/register.go beside repo.counts, capability letter o, ops.Anonymous, private branches filtered by the same seesPrivate(ctx) as everything else.

repo.activity — GET /api/v1/r/{repo}/activity

In:  Repo    string
     Branch  string  // "" = all
     User    string  // "" = all; set by the person page
     Since   string  // ISO date; "" = server picks the window (below)
     Until   string  // ISO date; "" = now
Out: Since, Until string
     Authors  []Author  // ordered by authored check-ins, desc
     Paths    []string  // tree-axis order (§5); index = pathIdx
     Rows     []Row
     Touches  []Touch
     Window   string    // "picked" | "given"

Author: {Name string; Checkins int; Files int; Hue int; Shade int}
Row:    [day int, hour int, author int, merge 0|1, files int, hash string, branch string]
Touch:  [day int, path int, author int, inherited 0|1]

Rows are one per check-in in the window; files is the authored file count (0 for a pure merge; the inherited count is not in the row — a tooltip that wants it asks the check-in page). Touches are one per (check-in, leaf path) with inherited=1 for merge rows, so a client may draw ghosts the way the prototype does, and a client that wants only authored work drops them with one filter. day is days since Since.

Window rule when Since is empty: the latest 900 days, extended backwards until the window holds at least 2,000 check-ins or the history is exhausted. Small repositories see everything; sqlite sees ~2.5 years. The client sees which happened via Window.

repo.activity.days — GET /api/v1/r/{repo}/activity/days

In:  Repo string; Branch string
Out: First, Last string        // ISO dates
     Authors []Author          // whole-history ranking, same shape
     Days    map[string]Day    // key ISO date; absent = no check-ins
     Releases map[string]string // ISO date → tag label
Day: {N int; Merges int; Top [][2]int /* [author, n] × ≤3 */; Authors int; Path int}

Whole history, always. Top is by authored check-ins; Path is the most touched top-level directory that day (the slice's tooltip). On sqlite this is 36,853 rows folded to ~6,000 days.

Both ops compute hue and shade server-side (§6) so that every renderer — Go SVG, browser canvas, a third-party client — colours a person identically.

5. Path order on the tree axis

Paths sorted by (dirname, lowercase path) so a directory's files are contiguous; directories with ≥ 5 paths in the window get a labelled band. This is the prototype's order and it reads well on all three stores; it is an op-level decision (the Paths array is already sorted) so both renderers agree.

A path gets a row of its own only when someone worked on it as a file: a non-merge check-in of fewer than 40 paths touched it in the window. Every other path — touched only by merges, or only by sweeps (a check-in of 40 or more paths: a vendored import, a licence re-stamp, a mass rename) — folds into one row per directory, named dir/, which sorts first in its directory — rolled up to the highest directory with nothing authored beneath it, so a vendored tree is one row rather than one per subdirectory. A check-in touching folded files is one touch on that row, inherited if it was a merge. Without the fold one zlib upgrade claims three hundred rows, and on the SVG (authored touches only) those rows are blank bands that push the real picture off the screen. Counts are untouched: the sweep's files still says how many paths it changed. Paths someone did edit keep their row, and a sweep's or merge's touch of such a path stays on it.

6. Colour is who, and nothing else

hue(name) = fold(h*31 + byte) mod 360 over the login's bytes. Among the top 24 authors, hues within 14° of each other get stepped lightness (Shade 0..3) so neighbours stay distinct. Saturation and base lightness are theme tokens (--dot-s, --dot-l: L 44% on light, 64% on dark). Hue is never used for a type, state, or file kind; every other mark (merge, release, inherited) is shape or alpha. Both renderers take Hue/Shade from the op.

7. Rendering

Server SVG, always. internal/web renders the core sample (either axis) and the slice as inline SVG from the op's Out. Circles carry <title> for hover; the controls are links (?axis=tree, ?window=all). This is the no-JavaScript state and the first paint for everyone (Principle 3).

Nothing is written on the plot. No directory labels, no date rail text, no hour or month labels, no month or year labels on the slice — the bands, the rules and the ticks stay, because they are the structure, and the hover carries every name and date they used to spell out. The year names itself in its numbers line beside the grid.

The person filter singles out; it never subtracts. ?user=NAME does not narrow the query: everyone's rows come back, the chosen person is drawn in their colour and everyone else goes grey (.au.mine on their group, the sheet greying its siblings). The marginals keep the project's grey bars and add the person's own over them, on the project's scale, so the pair reads as a share. On the slice a day they were not on is a ghost cell and a day they were is their colour at their own volume, with no second-author stripe; the ownership bar keeps only their segment in colour. A picture that deleted everyone else would lose its context and its colours, which is the whole point of having them.

The legend is the chooser. Under the picture, a wrapping row naming everyone in the window (up to 24), most check-ins first: a colour dot, the name, the check-in count, the whole entry a link that morphs the view. The legend greys with the plot — under a filter every other entry's dot goes muted and only the chosen one, marked and bold, keeps its colour — and a final everyone entry clears the filter.

Controls: axis, window, reset zoom, and nothing else. The branch chooser is gone (branch is always all) and so is the with merges toggle (inherited touches never drew). The window is recent (the server-picked window) or all (the first check-in to the last).

The tree axis runs time down the page and files across: one narrow column per path, one row per day with the newest at the top, in a block of fixed height (56 + 520 + 14) whatever the repository — a thousand paths thin the columns, they do not stretch the picture. Directories are vertical bands; the dates are a rail of rules down the left, stepping by month, or by year past a 900-day window. The marginals follow the axes: which files are touched most across the top, how busy each day was down the right. Only authored touches are drawn: what a merge carried in is shape, not anyone's work, and nothing on the page asks for it.

Size budget: a 900-day clock axis on sqlite is ~5,000 circles (~120 KB gzipped); the tree axis at the same window is ~25,000 (~600 KB), and ?window=all puts both axes over it. Past 1 MB the picture is drawn coarser, not shorter: a day bucket k doubles (1, 2, 4, … 32) and marks collapse to one per (bucket, row, person) cell drawn at the bucket's centre, with data-bucket="k" on the root and drawn at k-day resolution in the page. The window itself is never cut — a shortened window is a different question answered. The canvas draws the JSON at full resolution regardless, so the coarsening is the no-JS fallback only.

Canvas, when JavaScript runs. A small script (no framework; Datastar drives the controls) fetches repo.activity as JSON and replaces the SVG with a canvas: hover tooltips, marginal hit-tests that name files and hours, drag-zoom of the day range, the same greying under a person filter. Zoom state is JavaScript-only, not in the URL — a drag is exploration, not a place; a reset zoom button appears while zoomed (Escape and double-click do the same).

The slice is SVG only. One block per year: a column per week, a row per weekday, 11 px cells on a 13 px step. Every day of the year is a cell — one with no work in it is drawn in --slice-empty, so the year reads as a whole shape — and nothing is ever drawn inside a cell: the colour is the day's leading person and the opacity alone is the weight, from 0.25 at one check-in to solid at the year's own busy days, on a log ramp per year so the middle of a year does not pin at the top. Releases are a small marker in the gutter above the grid rather than a notch competing with the cell it sits in; there is no second-author stripe. Under a person filter their days keep their colour at their own weight, a day that was someone else's is a grey cell at the day's weight, and an empty day stays empty — the rest of the project stays visible behind the person. To the right, a 150 px ownership bar and the year's numbers, the first line of which names the year — that line is what carries the comparison between years. The page shows the last eight years and links to ?years=all; the home page shows one block. It is small enough that canvas buys nothing.

8. Doors

9. Caching and cost

10. Accessibility

The SVG carries an aria-label sentence per chart ("3,412 check-ins by 14 people, 2024-03-01 to 2026-08-30; drh led 62% of days") and a <title> per mark. The marginal bars and the per-year numbers are also emitted as a visually hidden table so a screen reader gets the summary the sighted reader gets from the shape. Colour is never the only carrier: the legend pairs each swatch with a name and count, merges are hollow, releases are notches.

11. Privacy and roles

Nothing new is exposed. Every row is a check-in the caller could already see on the timeline; private branches are filtered by seesPrivate before aggregation so counts cannot leak the existence of a hidden branch. Login names are the ones the timeline shows. Hours are the check-in's own UTC timestamp; quarry does not guess a time zone.

12. Plan

Three landings, each landable alone.

# lands touches proves it
1 index columns, both ops, docs/api.md internal/index, internal/api/register.go index tests for the merge rule on a hand-built DAG (a merge's files inherited, its check-in counted); op tests for window rule and branch/user filters; the benchmark
2 server SVG for both views; activity page; doors internal/web web tests asserting structure — circle count equals row count, data-author hooks, no-JS controls are links; no copy pins
3 canvas enhancement internal/web/assets browser check that the canvas replaces the SVG and hover/zoom work; the SVG remains the no-JS state

Person-page and home-page doors ride with 2. Landing 1 needs an index rebuild on deployed repositories; IndexBehind already tells the page when that has not happened.

Drawbacks

Rationale and alternatives

Prior art

GitHub's contribution graph and punch card; GitLab's contributor analytics; Gitea/Forgejo's activity page; Fossil's /reports (whose by user / by file tables are the nearest ancestor, and whose merge counting has the same inflation). Outside forges: the core sample metaphor is from geology — a column pulled from the ground with every layer in place — and the slice borrows the calendar-row layout of the contribution graph while replacing its one channel (volume) with three (who, how much, what).

Unresolved questions

Future possibilities