Artifact [1cb36af6a8]

Artifact 1cb36af6a861ca9a0dc261796381d450f98184c4f7e4da073cf2cbb1a6476dd2:

• File GENERATING-RFCS-FROM-SOURCE.md — part of check-in [64deb0b261] at 2026-07-08 03:50:51 on branch trunk — Add multi-document suite proving lessons to RFC generation runbook

  Document the two lessons from taking a multi-document protocol suite through clean-room proof: use an independent multi-reviewer panel (the verdict is a sample, not a theorem), and add a joint cross-document proof (per-document proofs can pass in isolation while the seam between them stays broken). Tie both into the clean-room invariant and the success criteria. (user: dmestas, size: 18314) [annotate] [blame] [check-ins using]

# Generating RFCs from Existing Source Code — A Cloner's Runbook

This repository is a **neutral, source-agnostic RFC authoring workflow**. Given a body of source
code and its documentation, it produces publishable RFC-style Internet-Drafts while keeping a
separate, auditable evidence chain that proves every public claim traces back to the source.

This runbook is the end-to-end path a fresh cloner follows to go from an empty checkout to
rendered RFC drafts. It uses **rsync** as the worked example throughout, but nothing here is
rsync-specific — swap in any codebase with a protocol, format, or data model worth specifying.

> **Mental model.** The repo root (`suite-discovery.contract.yaml`, `workflow.contract.yaml`, `core/`, `briefs/`, `toolchains/`, `templates/`, `prompts/`, `AGENTS.md`) is **reusable and never edited per project**.
> Discovery happens first against a staged source corpus.
> Everything source-specific then lives in one **project pack** under `projects/<name>/`.
> You run suite discovery to decide the document breakdown, scaffold a pack from that decision, then run the drafting workflow against the pack.

---

## 0. Prerequisites

Verify these once, before any run:

- **Render toolchain:** `kramdown-rfc` and `xml2rfc` on `PATH` (they pull in Ruby and Python).
  A draft is not "done" until it renders `.md → .xml → .txt` cleanly.
- **A multi-agent harness.** The workflow is harness-neutral; you translate it into whatever your
  harness provides (an orchestrator that can dispatch stage-specific sub-agents, gate on their
  output, and loop). A deterministic multi-agent orchestrator ("ultracode"-style) is the natural fit
  because the contract is a staged graph with gates and recursive correction.
- **The authoritative source corpus available locally** (see Step 2). The workflow refuses to draft
  public prose from memory — it drafts from source you have staged.

---

## 1. Obtain the authoritative corpus

Get the ground truth onto local disk first — suite discovery and every later claim depend on it,
with no network dependency at run time.

- **Clone/copy the source tree** and the shipped docs that live with it (manpages, protocol notes,
  schemas). Optionally convert bundled HTML/other docs to clean prose for lookup.
- Pin the exact version + commit you will cite.

*(rsync example: the reference source at a pinned commit, its shipped manpages, and the
daemon-protocol note.)*

---

## 2. Discover the RFC suite shape  ← the pivotal pre-pack step

**Run the discovery workflow** — not just the prompt in isolation:

- `suite-discovery.contract.yaml`
- `schemas/suite-discovery-contract.schema.json`
- `templates/NORMATIVE-SURFACE-INVENTORY.md`
- `templates/SUITE-BOUNDARY-DECISION.md`
- `templates/PACK-SCAFFOLD-INPUTS.md`
- `templates/SUITE-DISCOVERY-BRIEF.md`
- `prompts/discover-rfc-suite-shape.md`

This happens *before* any pack is scaffolded. Its job is **discovery and suite design only**: no drafting, no file scaffolding, no repo tree emission.

The contract fixes a five-stage pre-pack workflow with explicit gates and outputs:

1. **`discovery-scope`** → lock the system name, corpus boundary, observed surfaces, authority rule, and discovery-only objective.
2. **`corpus-map`** → separate authoritative, informative, and observed materials; mark contradictions and gaps.
3. **`surface-inventory`** → fill a `normative-surface-inventory` artifact covering the real algorithms, processes, formats, protocols, roles, and terminology domains.
4. **`suite-shaping`** → turn that inventory into a `suite-boundary-decision` that merges weak candidates, states non-goals, assigns dependencies, and fixes terminology ownership.
5. **`handoff-packaging`** → package the final `suite-discovery-brief` and `pack-scaffold-inputs` artifacts for downstream scaffolding.

The decision rule is the same throughout: prefer the **smallest clean RFC suite** that preserves clean ownership, clean-room implementability, reduced duplication, reviewable boundaries, and reuse of shared format/protocol layers. Subordinate material such as deployment guidance, UI behavior, helper code, or illustrative-only fixtures stays merged, dependent, or out of scope unless it exposes an independent normative surface.

The final handoff is a **suite-discovery brief** in six sections, ending in a **`Pack scaffold inputs`** block whose headings map one-to-one onto the pack generator's expected inputs. That final block is what carries the suite decision into Step 3. Prepare the inputs first: the authoritative corpus, any informative/background material, observed runtime surfaces, and the authority rule to apply when sources disagree — or an explicit `unknown` for discovery to infer and justify.

> **Why this step is load-bearing.** The suite shape is what makes the output *implementable* rather than merely descriptive. A protocol split along its real layers — a foundational wire/format document plus the dependent documents that cite it — lets each document be specified to the byte and proven buildable on its own. Getting the breakdown right here, before a single pack file is scaffolded, is the difference between "an overview of how it behaves" and "a spec you can build against."

*(rsync example: discovery returns a two-document suite — a **foundational Wire Transport & Daemon Protocol** document (session setup, handshake, auth, version/digest negotiation, multiplex framing, shared integer encodings) and a dependent **File List & Delta Transfer Protocol** document (file-list encoding, checksum negotiation, sums header, rolling/strong checksums, token stream, reconstruction) that cites the wire document for framing/encodings — plus the non-goals kept out of the suite.)*

---

## 3. Scaffold the pack from the discovery handoff, and stage the corpus

Feed the discovery workflow's final **`Pack scaffold inputs`** block — the one shaped by `templates/PACK-SCAFFOLD-INPUTS.md` and emitted inside the `suite-discovery-brief` — into **`projects/PROMPT-GENERATE-PROJECT-PACK.md`**, using `projects/template-ietf/` as the reusable IETF starter. This produces `projects/<name>/` already reflecting the discovered breakdown:

- one `briefs/<doc>.md` per canonical document (scope in/out, normative heart, required sections, clean-room capabilities, review traps);
- `suite-brief.md` — suite charter + cross-document structure + vocabulary ownership;
- `corpus-brief.md` — per-document section-to-source routing;
- `references.md` — internal dependency map + terminology ownership + any external normative references the algorithms require (e.g. the digest specs a checksum is computed against);
- `draft-metadata.md` — per-document titles, abstract anchors, stream/status (author blocks stay human-input placeholders — do not invent authors);
- `README.md`, plus the `references/` and `runs/` directories.

Then **stage the corpus into the pack** so claims anchor to a real `file:function` with no run-time network dependency:

- copy the source tree from Step 1 into `projects/<name>/references/<src>/`;
- write `references/source-locator.md` (version + commit, key wire/format constants, and the section-to-file routing map the briefs cite);
- confirm the authority order in `corpus-brief.md`: *implementation + shipped docs win over explanatory prose (FAQ/wiki/blog), which is informative only.*

*(rsync example: `projects/rsync/` with `briefs/wire-transport.md` + `briefs/delta-transfer.md`, the source staged under `references/rsync-src/`, and a `source-locator.md` mapping mux framing → `io.c`, file-list encoding → `flist.c`, checksums → `checksum.c`.)*

---

## 4. Verify run prerequisites for this pack

Before translating/running, confirm:

- render toolchain present (Step 0);
- the corpus is local and its **version matches what you will cite** (pin the commit);
- **if you will capture observed-runtime examples, the runtime binary must match the corpus
  version** — otherwise mark runtime examples *illustrative-only* and keep normative claims anchored
  to source. (Byte-level examples are constructed from the accepted claims, not from a live capture.)

---

## 5. Translate the workflow for your harness

Run **`prompts/translate-workflow-for-harness.md`** with concrete run facts:

- selected project pack (`projects/<name>/`),
- target scope: `document:<brief>`, `subset:<brief>,<brief>`, or `suite:all`,
- a dedicated writable run root (`projects/<name>/runs/<run-id>/`),
- the verified prerequisites from Step 4.

It loads `AGENTS.md`, `workflow.contract.yaml`, `core/`, the reusable materials, and your pack, then
emits a **native primitive map** (controller, stage-execution unit, gate mechanism, evidence/public
workspaces, recursive-correction routing, escalation path) and a **ready-to-run handoff**.

> You do **not** modify `workflow.contract.yaml` — it already specifies stage order, gates, the
> public/evidence split, and recursive correction. Translation maps those semantics onto your
> harness's primitives; it does not redesign them.

---

## 6. Run the workflow

Run **`prompts/run-selected-project.md`** with the handoff. Execution follows the contract's stage
graph. The 15 stages, in order, with their gates:

| # | Stage | Produces (family) | Gate blocks until |
|---|---|---|---|
| 1 | scope | run-brief (evidence) | objective, deliverables, exclusions, authority order set |
| 2 | context-packet | context-packet (evidence) | vocabulary/constraints/open-questions assembled (orientation ≠ authority) |
| 3 | source-map | source-map (evidence) | source units named with retrieval handles; gaps marked |
| 4 | source-scout | source-scout-report (evidence) | narrow source reads answered with exact anchors; evidence vs inference separated |
| 5 | claim-ledger | claim-ledger (evidence) | every claim has authority class + anchor + status + section impact |
| 6 | capability-matrix | capability-matrix (evidence) | each required capability = covered/open/scoped-out/blocking |
| 7 | draft-skeleton | section-map + **public draft source** | sections map to accepted claims only (no prose from memory) |
| 8 | example-lab | example-report (evidence) | examples validated-from-observed or explicitly illustrative-only |
| 9 | mechanical-validation | **rendered public output** + mechanical-report | kramdown-rfc → xml2rfc renders clean; warnings vs blockers dispositioned |
| 10 | review-gauntlet | review-report (evidence) | no blocker findings across grounding/security/example/provenance lanes |
| 11 | reader-onramp | reader-report (evidence) | newcomer on-ramp judged from rendered public text alone |
| 12 | prose-audit | prose-report + cleanroom-packet | wording risks dispositioned without changing semantics |
| 13 | correction-loop | correction-log / escalation-log | (only on failure — see below) |
| 14 | clean-room-proof | cleanroom-report (evidence) | a fresh, non-drafting reviewer can recover the required capability **from the public packet alone** |
| 15 | final-packaging | **public-manifest** + evidence-manifest | deliverables complete; full proof chain retained |

Running invariants the harness must enforce:

- **Source-grounded only.** Public technical/normative claims enter the public track *only* from
  accepted claim records. Notes before prose.
- **Adversarial verification.** After the claim ledger, independent skeptics try to *refute* each
  claim against the source; any claim whose anchor doesn't hold is withheld from public prose.
- **Public/evidence separation + provenance containment.** Public text carries the spec only — no
  source paths, claim IDs, tool/agent names, or "derived-from-source" statements. Evidence carries
  the anchors and proof and stays run-local.
- **Recursive correction.** When a gate fails, route the fix to the **earliest faulty artifact**,
  re-run only the affected downstream stages, and **re-prove clean-room** — a bounded loop, not a
  one-shot patch. Genuine design decisions (cross-document ownership, anything changing
  scope/semantics/publication) **escalate to a human and halt promotion** rather than being guessed.
- **Clean-room is the bar.** For a protocol document, "PASS" means a fresh implementer could build an
  interoperable peer from the public text alone. Unresolved blockers stop promotion. Prove it with an
  independent **multi-reviewer panel**, not a single reviewer — the verdict is a sample, not a theorem
  (see "Proving a multi-document suite" below).

Scope tip: run `suite:all` (or a `subset:`) so coupled documents are produced together; a foundational
document should be corrected before the documents that cite it.

---

## 7. Collect and package outputs

By the end of a run, `projects/<name>/runs/<run-id>/` contains:

- `public/<doc>/draft.{md,txt,xml}` per document + `public/public-manifest.md`
- `evidence/…` — the full proof chain (claim ledgers, scout reports, matrices, review/clean-room
  reports, correction/escalation logs, `evidence-manifest.md`)
- `gates/` — per-stage PASS/FAIL records

**To share:** bundle the **public** deliverables (the drafts + public manifest). **Exclude the
evidence tree** — it contains source anchors and internal identifiers that provenance containment
keeps out of anything published. Ship the evidence bundle separately only for audit.

Render/validate any draft locally:

```
kramdown-rfc <draft>.md > <draft>.xml
xml2rfc --text <draft>.xml -o <draft>.txt
```

---

## What "success" looks like

- Every selected document renders cleanly.
- Each document's clean-room proof is **PASS** (a fresh implementer could build it) — **or** it is
  honestly **BLOCKED** with an enumerated punch-list of remaining gaps and any open design decisions.
  A blocked document with a precise gap list is a legitimate, useful output; a false PASS is the only
  real failure.
- A shareable public bundle + a retained evidence audit trail.
- Proven with an independent **multi-reviewer panel**, and (for a suite) a **joint cross-document
  proof** — a single-reviewer PASS is not reliable. See "Proving a multi-document suite" below.

---

## Proving a multi-document suite

Two lessons that only surface once you actually run a multi-document protocol suite through clean-room
proof. Both change how you should run Steps 6–7.

### Lesson 1 — the clean-room verdict is a sample, not a theorem; use a panel

A single clean-room reviewer's PASS is not reliable. Reviewers are competent but their coverage
varies: **the same unchanged public text can pass one reviewer pair and then fail a fresh pair** that
catches a real gap the first missed (this is observed, not hypothetical).

- Prove with a **multi-reviewer panel** — 2 minimum, **3+ for a verdict you will act on** — each
  **packet-only** (only the rendered public text; no source, no evidence, no briefs beyond the
  reviewer role) and at **different implementation angles**: initiator vs. responder, sender vs.
  receiver, plus an adversarial third angle (a byte-alignment auditor, or a "walk a concrete
  multi-file transfer" auditor) pointed at exactly the seam you trust least.
- A document passes only if a **majority (ideally all)** independently succeed.
- **Never launder a majority-with-dissent into an unqualified PASS.** If one competent reviewer flags a
  genuine blocker, close it — don't outvote it. (Also watch your own tooling — a substring match on
  `BLOCKING` will happily fire on `NON-BLOCKING`; read the actual findings.)
- **State honestly what PASS means** in the shipped status: *an independent panel could build a
  conforming peer from the text* — strong empirical evidence, **not** a formal guarantee of zero gaps.
  Enumerate residual non-blocking items and tell readers to verify against a real implementation.

### Lesson 2 — per-document proofs pass in isolation; the seam between them does not

In a suite where document B depends on document A, **each document can pass its own clean-room proof
while the join between them is still broken** — B delegates a mechanism to A ("see [A] for the seed
exchange") that A never actually specifies. Per-document loops cannot catch this.

- **Add a joint cross-document proof** as an explicit step: B's reviewers must read A's *final* text
  (not a mid-run snapshot) so delegated citations are checked end-to-end.
- **Absorb delegations in dependency order.** If B is corrected *after* A is frozen, B's new
  "see [A] for X" delegations never land in A. Reconcile by collecting every delegation the dependents
  make, adding those to the depended-on document, *then* re-proving both against each other's final
  text.
- **Distinguish a structural gap from a fillable one.** When a needed mechanism is owned by *neither*
  document (e.g. the transfer-loop / file-indexing choreography that binds each file's data to its
  file-list entry), no amount of field-filling converges — that is a **decomposition signal**. Give
  the layer an explicit owner (a new section in the closest document, or a new document) and re-run.
  Looping on a structural gap just replateaus.
- **Expect convergence in layers.** Each proof round peels back to the next stratum; a foundational
  document may pass, then a dependent's proof exposes what the foundation still owes it. Bound the loop
  and detect plateaus so you stop and re-decompose (or escalate) instead of grinding indefinitely.

## One-screen checklist
1. Verify `kramdown-rfc` + `xml2rfc` (Step 0).
2. Obtain the source tree + shipped docs locally; pin the commit (Step 1).
3. Run the discovery workflow — `suite-discovery.contract.yaml`, its schema, the discovery templates, and `prompts/discover-rfc-suite-shape.md` — to produce a `suite-discovery-brief` plus final `Pack scaffold inputs` block (Step 2).
4. Feed that `Pack scaffold inputs` block into `projects/PROMPT-GENERATE-PROJECT-PACK.md` (starter: `projects/template-ietf/`); stage the corpus under `references/` + `source-locator.md` + authority order (Step 3).
5. Confirm corpus/runtime version match (Step 4).
6. `prompts/translate-workflow-for-harness.md` → handoff (Step 5).
7. `prompts/run-selected-project.md` → run the staged graph with gates + recursive correction (Step 6).
8. Package public drafts to share; keep evidence separate (Step 7).