Artifact 38b65381a98220d499064733321fb74fd990b0a8c181b8029d8487519040b816:
- File
conformance/doc.go
— part of check-in
[57a558fcbf]
at
2026-10-10 23:52:34
on branch trunk
— test(conformance): harness for an RFC conformance suite (#262) (#264)
Adds the conformance package that the per-draft suites build on. Nothing in the repository has yet checked go-libfossil against the Fossil drafts requirement by requirement.
- A Case names its requirement. A Gap (an issue) or Deviation (a reason) marks a known difference. A marked case still runs: while it fails it is skipped, and once it conforms it fails until the mark is removed, so the list of known failures cannot go stale. - Requirements tables pin the SHA-256 of the drafts and errata they were written against. Those are copied unmodified into conformance/rfcs by scripts/refresh-conformance-rfcs.sh. A changed draft fails the coverage test until its table has been re-read and its pin moved. - Coverage is derived from the cases rather than recorded: - COVERED: every case conforms. - PARTIAL: the table's row says what its cases leave out. - DEVIATION: a case is marked as a deliberate difference. - FAILING: a case is marked with an open gap. - GAP: no case yet.
CONFORMANCE_REPORT=<path> writes the requirement-by-requirement report as Markdown. - fossil(1) is run on every case's input, through test-parse-manifest, so a case's expectation is checked against fossil each time the suite runs. withZ appends the Z card, so a case that edits one card never also breaks the checksum.
The harness's own rules are tested in harness_self_test.go. No suites are registered yet; the artifact-format suite is next (#263). (user: [email protected], size: 1300) [annotate] [blame] [check-ins using]
// Package conformance checks go-libfossil against the Fossil Internet-Drafts // in rfcs/, requirement by requirement. // // The package holds no library code; everything is in its tests. Each draft // has a requirements table in requirements/, written from the draft text, and // a test file whose cases name the requirement they exercise. Coverage is // derived from the cases, never recorded by hand: a requirement with no case // is a GAP, one whose cases all conform is COVERED (or PARTIAL when its table // row says what the cases leave out), and one with a case that is known to // fail is FAILING until the issue that case names is fixed. // // What counts as correct, in order: // // 1. The draft text, with the errata in rfcs/erratum. // 2. fossil(1), run on the same input wherever the draft is silent or an // erratum is unsettled. A case that expects something fossil(1) does not // do must say why. // 3. zit and libfossil, whose own suites inform which cases to write. // // A case that fails is marked with the issue tracking the fix and is skipped; // once it passes, it fails until the mark is removed, so the list of known // failures cannot go stale. Set CONFORMANCE_REPORT to a file path to have the // coverage test write a Markdown report there. package conformance