Generating RFCs from Existing Source Code — A Cloner's Runbook

This repository is a neutral, source-agnostic RFC authoring workflow. Given a body of source code and its documentation, it produces publishable RFC-style Internet-Drafts while keeping a separate, auditable evidence chain that proves every public claim traces back to the source.

This runbook is the end-to-end path a fresh cloner follows to go from an empty checkout to rendered RFC drafts. It uses rsync as the worked example throughout, but nothing here is rsync-specific — swap in any codebase with a protocol, format, or data model worth specifying.

Mental model. The repo root (suite-discovery.contract.yaml, workflow.contract.yaml, core/, briefs/, toolchains/, templates/, prompts/, AGENTS.md) is reusable and never edited per project. Discovery happens first against a staged source corpus. Everything source-specific then lives in one project pack under projects/<name>/. You run suite discovery to decide the document breakdown, scaffold a pack from that decision, then run the drafting workflow against the pack.

---

0. Prerequisites

Verify these once, before any run:

• Render toolchain: kramdown-rfc and xml2rfc on PATH (they pull in Ruby and Python). A draft is not "done" until it renders .md → .xml → .txt cleanly.
• A multi-agent harness. The workflow is harness-neutral; you translate it into whatever your harness provides (an orchestrator that can dispatch stage-specific sub-agents, gate on their output, and loop). A deterministic multi-agent orchestrator ("ultracode"-style) is the natural fit because the contract is a staged graph with gates and recursive correction.
• The authoritative source corpus available locally (see Step 2). The workflow refuses to draft public prose from memory — it drafts from source you have staged.

---

1. Obtain the authoritative corpus

Get the ground truth onto local disk first — suite discovery and every later claim depend on it, with no network dependency at run time.

• Clone/copy the source tree and the shipped docs that live with it (manpages, protocol notes, schemas). Optionally convert bundled HTML/other docs to clean prose for lookup.
• Pin the exact version + commit you will cite.

(rsync example: the reference source at a pinned commit, its shipped manpages, and the daemon-protocol note.)

---

2. Discover the RFC suite shape ← the pivotal pre-pack step

Run the discovery workflow — not just the prompt in isolation:

• suite-discovery.contract.yaml
• schemas/suite-discovery-contract.schema.json
• templates/NORMATIVE-SURFACE-INVENTORY.md
• templates/SUITE-BOUNDARY-DECISION.md
• templates/PACK-SCAFFOLD-INPUTS.md
• templates/SUITE-DISCOVERY-BRIEF.md
• prompts/discover-rfc-suite-shape.md

This happens before any pack is scaffolded. Its job is discovery and suite design only: no drafting, no file scaffolding, no repo tree emission.

The contract fixes a five-stage pre-pack workflow with explicit gates and outputs:

1. discovery-scope → lock the system name, corpus boundary, observed surfaces, authority rule, and discovery-only objective.
2. corpus-map → separate authoritative, informative, and observed materials; mark contradictions and gaps.
3. surface-inventory → fill a normative-surface-inventory artifact covering the real algorithms, processes, formats, protocols, roles, and terminology domains.
4. suite-shaping → turn that inventory into a suite-boundary-decision that merges weak candidates, states non-goals, assigns dependencies, and fixes terminology ownership.
5. handoff-packaging → package the final suite-discovery-brief and pack-scaffold-inputs artifacts for downstream scaffolding.

The decision rule is the same throughout: prefer the smallest clean RFC suite that preserves clean ownership, clean-room implementability, reduced duplication, reviewable boundaries, and reuse of shared format/protocol layers. Subordinate material such as deployment guidance, UI behavior, helper code, or illustrative-only fixtures stays merged, dependent, or out of scope unless it exposes an independent normative surface.

The final handoff is a suite-discovery brief in six sections, ending in a Pack scaffold inputs block whose headings map one-to-one onto the pack generator's expected inputs. That final block is what carries the suite decision into Step 3. Prepare the inputs first: the authoritative corpus, any informative/background material, observed runtime surfaces, and the authority rule to apply when sources disagree — or an explicit unknown for discovery to infer and justify.

Why this step is load-bearing. The suite shape is what makes the output implementable rather than merely descriptive. A protocol split along its real layers — a foundational wire/format document plus the dependent documents that cite it — lets each document be specified to the byte and proven buildable on its own. Getting the breakdown right here, before a single pack file is scaffolded, is the difference between "an overview of how it behaves" and "a spec you can build against."

(rsync example: discovery returns a two-document suite — a foundational Wire Transport & Daemon Protocol document (session setup, handshake, auth, version/digest negotiation, multiplex framing, shared integer encodings) and a dependent File List & Delta Transfer Protocol document (file-list encoding, checksum negotiation, sums header, rolling/strong checksums, token stream, reconstruction) that cites the wire document for framing/encodings — plus the non-goals kept out of the suite.)

---

3. Scaffold the pack from the discovery handoff, and stage the corpus

Feed the discovery workflow's final Pack scaffold inputs block — the one shaped by templates/PACK-SCAFFOLD-INPUTS.md and emitted inside the suite-discovery-brief — into projects/PROMPT-GENERATE-PROJECT-PACK.md, using projects/template-ietf/ as the reusable IETF starter. This produces projects/<name>/ already reflecting the discovered breakdown:

• one briefs/<doc>.md per canonical document (scope in/out, normative heart, required sections, clean-room capabilities, review traps);
• suite-brief.md — suite charter + cross-document structure + vocabulary ownership;
• corpus-brief.md — per-document section-to-source routing;
• references.md — internal dependency map + terminology ownership + any external normative references the algorithms require (e.g. the digest specs a checksum is computed against);
• draft-metadata.md — per-document titles, abstract anchors, stream/status (author blocks stay human-input placeholders — do not invent authors);
• README.md, plus the references/ and runs/ directories.

Then stage the corpus into the pack so claims anchor to a real file:function with no run-time network dependency:

• copy the source tree from Step 1 into projects/<name>/references/<src>/;
• write references/source-locator.md (version + commit, key wire/format constants, and the section-to-file routing map the briefs cite);
• confirm the authority order in corpus-brief.md: implementation + shipped docs win over explanatory prose (FAQ/wiki/blog), which is informative only.

(rsync example: projects/rsync/ with briefs/wire-transport.md + briefs/delta-transfer.md, the source staged under references/rsync-src/, and a source-locator.md mapping mux framing → io.c, file-list encoding → flist.c, checksums → checksum.c.)

---

4. Verify run prerequisites for this pack

Before translating/running, confirm:

• render toolchain present (Step 0);
• the corpus is local and its version matches what you will cite (pin the commit);
• if you will capture observed-runtime examples, the runtime binary must match the corpus version — otherwise mark runtime examples illustrative-only and keep normative claims anchored to source. (Byte-level examples are constructed from the accepted claims, not from a live capture.)

---

5. Translate the workflow for your harness

Run prompts/translate-workflow-for-harness.md with concrete run facts:

• selected project pack (projects/<name>/),
• target scope: document:<brief>, subset:<brief>,<brief>, or suite:all,
• a dedicated writable run root (projects/<name>/runs/<run-id>/),
• the verified prerequisites from Step 4.

It loads AGENTS.md, workflow.contract.yaml, core/, the reusable materials, and your pack, then emits a native primitive map (controller, stage-execution unit, gate mechanism, evidence/public workspaces, recursive-correction routing, escalation path) and a ready-to-run handoff.

You do not modify workflow.contract.yaml — it already specifies stage order, gates, the public/evidence split, and recursive correction. Translation maps those semantics onto your harness's primitives; it does not redesign them.

---

6. Run the workflow

Run prompts/run-selected-project.md with the handoff. Execution follows the contract's stage graph. The 15 stages, in order, with their gates:

#	Stage	Produces (family)	Gate blocks until
1	scope	run-brief (evidence)	objective, deliverables, exclusions, authority order set
2	context-packet	context-packet (evidence)	vocabulary/constraints/open-questions assembled (orientation ≠ authority)
3	source-map	source-map (evidence)	source units named with retrieval handles; gaps marked
4	source-scout	source-scout-report (evidence)	narrow source reads answered with exact anchors; evidence vs inference separated
5	claim-ledger	claim-ledger (evidence)	every claim has authority class + anchor + status + section impact
6	capability-matrix	capability-matrix (evidence)	each required capability = covered/open/scoped-out/blocking
7	draft-skeleton	section-map + public draft source	sections map to accepted claims only (no prose from memory)
8	example-lab	example-report (evidence)	examples validated-from-observed or explicitly illustrative-only
9	mechanical-validation	rendered public output + mechanical-report	kramdown-rfc → xml2rfc renders clean; warnings vs blockers dispositioned
10	review-gauntlet	review-report (evidence)	no blocker findings across grounding/security/example/provenance lanes
11	reader-onramp	reader-report (evidence)	newcomer on-ramp judged from rendered public text alone
12	prose-audit	prose-report + cleanroom-packet	wording risks dispositioned without changing semantics
13	correction-loop	correction-log / escalation-log	(only on failure — see below)
14	clean-room-proof	cleanroom-report (evidence)	a fresh, non-drafting reviewer can recover the required capability from the public packet alone
15	final-packaging	public-manifest + evidence-manifest	deliverables complete; full proof chain retained

Running invariants the harness must enforce:

• Source-grounded only. Public technical/normative claims enter the public track only from accepted claim records. Notes before prose.
• Adversarial verification. After the claim ledger, independent skeptics try to refute each claim against the source; any claim whose anchor doesn't hold is withheld from public prose.
• Public/evidence separation + provenance containment. Public text carries the spec only — no source paths, claim IDs, tool/agent names, or "derived-from-source" statements. Evidence carries the anchors and proof and stays run-local.
• Recursive correction. When a gate fails, route the fix to the earliest faulty artifact, re-run only the affected downstream stages, and re-prove clean-room — a bounded loop, not a one-shot patch. Genuine design decisions (cross-document ownership, anything changing scope/semantics/publication) escalate to a human and halt promotion rather than being guessed.
• Clean-room is the bar. For a protocol document, "PASS" means a fresh implementer could build an interoperable peer from the public text alone. Unresolved blockers stop promotion. Prove it with an independent multi-reviewer panel, not a single reviewer — the verdict is a sample, not a theorem (see "Proving a multi-document suite" below).

Scope tip: run suite:all (or a subset:) so coupled documents are produced together; a foundational document should be corrected before the documents that cite it.

---

7. Collect and package outputs

By the end of a run, projects/<name>/runs/<run-id>/ contains:

• public/<doc>/draft.{md,txt,xml} per document + public/public-manifest.md
• evidence/… — the full proof chain (claim ledgers, scout reports, matrices, review/clean-room reports, correction/escalation logs, evidence-manifest.md)
• gates/ — per-stage PASS/FAIL records

To share: bundle the public deliverables (the drafts + public manifest). Exclude the evidence tree — it contains source anchors and internal identifiers that provenance containment keeps out of anything published. Ship the evidence bundle separately only for audit.

Render/validate any draft locally:

kramdown-rfc <draft>.md > <draft>.xml
xml2rfc --text <draft>.xml -o <draft>.txt

---

What "success" looks like

• Every selected document renders cleanly.
• Each document's clean-room proof is PASS (a fresh implementer could build it) — or it is honestly BLOCKED with an enumerated punch-list of remaining gaps and any open design decisions. A blocked document with a precise gap list is a legitimate, useful output; a false PASS is the only real failure.
• A shareable public bundle + a retained evidence audit trail.
• Proven with an independent multi-reviewer panel, and (for a suite) a joint cross-document proof — a single-reviewer PASS is not reliable. See "Proving a multi-document suite" below.

---

Proving a multi-document suite

Two lessons that only surface once you actually run a multi-document protocol suite through clean-room proof. Both change how you should run Steps 6–7.

Lesson 1 — the clean-room verdict is a sample, not a theorem; use a panel

A single clean-room reviewer's PASS is not reliable. Reviewers are competent but their coverage varies: the same unchanged public text can pass one reviewer pair and then fail a fresh pair that catches a real gap the first missed (this is observed, not hypothetical).

• Prove with a multi-reviewer panel — 2 minimum, 3+ for a verdict you will act on — each packet-only (only the rendered public text; no source, no evidence, no briefs beyond the reviewer role) and at different implementation angles: initiator vs. responder, sender vs. receiver, plus an adversarial third angle (a byte-alignment auditor, or a "walk a concrete multi-file transfer" auditor) pointed at exactly the seam you trust least.
• A document passes only if a majority (ideally all) independently succeed.
• Never launder a majority-with-dissent into an unqualified PASS. If one competent reviewer flags a genuine blocker, close it — don't outvote it. (Also watch your own tooling — a substring match on BLOCKING will happily fire on NON-BLOCKING; read the actual findings.)
• State honestly what PASS means in the shipped status: an independent panel could build a conforming peer from the text — strong empirical evidence, not a formal guarantee of zero gaps. Enumerate residual non-blocking items and tell readers to verify against a real implementation.

Lesson 2 — per-document proofs pass in isolation; the seam between them does not

In a suite where document B depends on document A, each document can pass its own clean-room proof while the join between them is still broken — B delegates a mechanism to A ("see [A] for the seed exchange") that A never actually specifies. Per-document loops cannot catch this.

• Add a joint cross-document proof as an explicit step: B's reviewers must read A's final text (not a mid-run snapshot) so delegated citations are checked end-to-end.
• Absorb delegations in dependency order. If B is corrected after A is frozen, B's new "see [A] for X" delegations never land in A. Reconcile by collecting every delegation the dependents make, adding those to the depended-on document, then re-proving both against each other's final text.
• Distinguish a structural gap from a fillable one. When a needed mechanism is owned by neither document (e.g. the transfer-loop / file-indexing choreography that binds each file's data to its file-list entry), no amount of field-filling converges — that is a decomposition signal. Give the layer an explicit owner (a new section in the closest document, or a new document) and re-run. Looping on a structural gap just replateaus.
• Expect convergence in layers. Each proof round peels back to the next stratum; a foundational document may pass, then a dependent's proof exposes what the foundation still owes it. Bound the loop and detect plateaus so you stop and re-decompose (or escalate) instead of grinding indefinitely.

One-screen checklist

1. Verify kramdown-rfc + xml2rfc (Step 0).
2. Obtain the source tree + shipped docs locally; pin the commit (Step 1).
3. Run the discovery workflow — suite-discovery.contract.yaml, its schema, the discovery templates, and prompts/discover-rfc-suite-shape.md — to produce a suite-discovery-brief plus final Pack scaffold inputs block (Step 2).
4. Feed that Pack scaffold inputs block into projects/PROMPT-GENERATE-PROJECT-PACK.md (starter: projects/template-ietf/); stage the corpus under references/ + source-locator.md + authority order (Step 3).
5. Confirm corpus/runtime version match (Step 4).
6. prompts/translate-workflow-for-harness.md → handoff (Step 5).
7. prompts/run-selected-project.md → run the staged graph with gates + recursive correction (Step 6).
8. Package public drafts to share; keep evidence separate (Step 7).