Sub-Manifests

Why flat manifests stop scaling

A Fossil baseline manifest carries one F card per file in the check-in — not per changed file. Manifest size is therefore linear in tree width. Any operation that reconstructs a check-in's complete file set performs work proportional to its file count; checkout, diff, crosslink, rebuild, annotate, and commit all need that enumeration.

The 2010 design note measured this on NetBSD's pkgsrc history: more than 60,000 files, manifests around 5 MB, and an eight-hour rebuild dominated by parsing roughly 190,000 manifests. The draft records a later measurement at roughly 103,000 files and 10.2 MB per manifest. The precise numbers changed; the shape did not. Work grew with the number of files in every check-in rather than with the handful of files changed.

The note proposed the remedy the draft makes precise: a tree of sub-manifests linked by a new S card. It supplied the direction; the draft fixes the card grammar, deterministic partition, tree walk, sync behavior, and deployment gates needed for independent implementations to agree. In a one-level tree a typical change rewrites the check-in manifest and one leaf; in a deeper tree it also rewrites the interior nodes on the path between them.

Figures 1–3 include compact card-order context. D and the final Z are required on these check-in manifests. R and U are shown to orient the file-set run within a representative manifest, not to imply that either card is universally required.

The complete card-stack diagrams use one convention throughout: a blue-gray header names the artifact type and, when useful, its abbreviated hash; every bordered card row represents one LF-terminated card; and rust marks a changed or exceptional row rather than a different card class. Compact algorithm diagrams may summarize a run, but never place two individual cards on the same text line.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.08
boxrad = 0.018

# Two complete baseline check-in manifests. D precedes the F run; Z is final.
H1: box "BASELINE MANIFEST" "91af...c302" wid 1.58 ht 0.48 fill 0xe7eef2 at 0,0
H2: box "BASELINE MANIFEST" "b74e...19d0" wid 1.58 ht 0.48 fill 0xe7eef2 at H1.x + 1.78,H1.y
text "CHECK-IN 1" at H1.x,H1.n.y + 0.22 color 0x53616a
text "CHECK-IN 2" at H2.x,H2.n.y + 0.22 color 0x53616a
D1: box "D  2026-09-13..." wid 1.58 ht 0.28 fill 0xf3f4f3 at H1.x,H1.s.y - 0.14
D2: box "D  2026-09-14..." same at H2.x,H2.s.y - 0.14

A1: box "F README  a1..." wid 1.58 ht 0.28 fill 0xf7f8f7 at H1.x,D1.s.y - 0.14
A2: box "F guide   b2..." same at H1.x,A1.s.y - 0.14
A3: box "F main    c3..." same at H1.x,A2.s.y - 0.14
A4: box "F tests   d4..." same at H1.x,A3.s.y - 0.14
B1: box "F README  a1..." wid 1.58 ht 0.28 fill 0xf7f8f7 at H2.x,D2.s.y - 0.14
B2: box "F guide   b2..." same at H2.x,B1.s.y - 0.14
B3: box "F main    e9..." same fill 0xf4e8e2 color 0x8b4a32 thickness 0.011 at H2.x,B2.s.y - 0.14
B4: box "F tests   d4..." same fill 0xf7f8f7 color 0x29333a thickness 0.006 at H2.x,B3.s.y - 0.14

R1: box "R  ..." wid 1.58 ht 0.26 fill 0xf3f4f3 at H1.x,A4.s.y - 0.13
U1: box "U  alice" same at H1.x,R1.s.y - 0.13
Z1: box "Z  ..." same fill 0xe7eef2 at H1.x,U1.s.y - 0.13
R2: box "R  ..." wid 1.58 ht 0.26 fill 0xf3f4f3 at H2.x,B4.s.y - 0.13
U2: box "U  alice" same at H2.x,R2.s.y - 0.13
Z2: box "Z  ..." same fill 0xe7eef2 at H2.x,U2.s.y - 0.13

line dashed from A1.e to B1.w color 0x8b949a
line dashed from A2.e to B2.w color 0x8b949a
arrow from A3.e to B3.w color 0x8b4a32 thickness 0.011
line dashed from A4.e to B4.w color 0x8b949a
text "D -> complete F run -> R -> U -> Z (final)" at (H1.x + H2.x)/2,Z1.s.y - 0.24 color 0x53616a
text "main changes; every F card is stored + parsed again: O(N)" at (H1.x + H2.x)/2,Z1.s.y - 0.56 color 0x29333a

What classic delta manifests already do

Older Fossil readers already understand two check-in encodings. A baseline manifest carries the complete sorted F run. A delta manifest carries a B card naming a baseline plus a sorted F run containing only additions, changes, and hashless deletions. The established merge produces the same effective complete file set.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.10
boxrad = 0.018

text "CHECK-IN 1" at -1.05,0 color 0x53616a
BH: box "BASELINE MANIFEST" "91af...c302" wid 1.62 ht 0.48 fill 0xe7eef2 at -1.05,-0.46
BD: box "D  2026-09-13..." wid 1.52 ht 0.27 fill 0xf3f4f3 at BH.x,BH.s.y - 0.135
BF1: box "F alpha    a1..." wid 1.52 ht 0.27 fill 0xf7f8f7 at BH.x,BD.s.y - 0.135
BF2: box "F beta     b2..." same at BH.x,BF1.s.y - 0.135
BF3: box "F delta    d4..." same at BH.x,BF2.s.y - 0.135
BR: box "R  ..." wid 1.52 ht 0.25 fill 0xf3f4f3 at BH.x,BF3.s.y - 0.125
BU: box "U  alice" same at BH.x,BR.s.y - 0.125
BZ: box "Z  ..." same fill 0xe7eef2 at BH.x,BU.s.y - 0.125

text "CHECK-IN 2" at 1.05,0 color 0x53616a
DH: box "DELTA MANIFEST" "b74e...19d0" wid 1.62 ht 0.48 fill 0xe7eef2 at 1.05,-0.46
DB: box "B  91af...c302" wid 1.62 ht 0.27 fill 0xf3f4f3 at DH.x,DH.s.y - 0.135
DD: box "D  2026-09-14..." same at DH.x,DB.s.y - 0.135
DF1: box "F beta   b9...  changed" wid 1.62 ht 0.27 fill 0xf4e8e2 color 0x8b4a32 thickness 0.011 at DH.x,DD.s.y - 0.135
DF2: box "F charlie c3...  added" wid 1.62 ht 0.27 fill 0xf4e8e2 color 0x8b4a32 thickness 0.006 at DH.x,DF1.s.y - 0.135
DF3: box "F delta       deleted" wid 1.62 ht 0.27 fill 0xf4e8e2 color 0x8b4a32 thickness 0.006 at DH.x,DF2.s.y - 0.135
DR: box "R  ..." wid 1.62 ht 0.25 fill 0xf3f4f3 at DH.x,DF3.s.y - 0.125
DU: box "U  alice" same at DH.x,DR.s.y - 0.125
DZ: box "Z  ..." same fill 0xe7eef2 at DH.x,DU.s.y - 0.125

M: box "existing delta-manifest merge" wid 2.25 ht 0.40 fill 0xe7eef2 at 0,DZ.s.y - 0.55
arrow from BZ.s to M.nw color 0x53616a
arrow from DZ.s to M.ne color 0x53616a
EH: box "effective complete sorted F run" wid 2.25 ht 0.38 fill 0xe7eef2 at M.x,M.s.y - 0.42
arrow from M.s to EH.n color 0x53616a
EF1: box "F alpha    a1..." wid 3.20 ht 0.28 fill 0xf7f8f7 at 0,EH.s.y - 0.14
EF2: box "F beta     b9..." same at 0,EF1.s.y - 0.14
EF3: box "F charlie  c3..." same at 0,EF2.s.y - 0.14

text "artifact-level B/F merge" "not blob compression; no S cards" at 0,EF3.s.y - 0.40 color 0x29333a

This is an artifact-level delta manifest, not the repository's lower-level blob delta compression. It also contains no S cards: B means “merge against this baseline,” whereas S means “this check-in's file set is stored in referenced tree nodes.”

Delta manifests can reduce the literal F-card text in the manifest artifact, but Fossil normally stores ordinary manifest artifacts using delta compression already, so the additional repository-storage saving is often negligible. More importantly, the consumer must still expand the baseline and walk the delta-manifest merge to recover the complete file set. The read and parse work therefore remains linear in tree width.

That linear reconstruction has a second consequence: flat and delta manifests expose no independently addressed slices of the file list. A change therefore cannot reuse an unchanged portion as a structural artifact. Existing file-content blobs may still be shared, and Fossil may delta-compress one stored artifact against another; neither avoids reconstructing the complete file set.

What a tree changes

Sub-manifests address that reconstruction and sharing boundary. A sub-manifest carries a contiguous slice of a check-in's sorted file set, either as F cards (a leaf) or as S cards referencing lower sub-manifests (interior). Because the slices are content-addressed, an unchanged slice is literally the same artifact across check-ins and is stored once.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.10
boxrad = 0.018

# Tree check-in card order plus leaf sub-manifest legality. Each snapshot is
# top-to-bottom with exactly one card per row; R/U are representative context.
text "CHECK-IN 1" at 0,0 color 0x53616a
OH: box "TREE CHECK-IN MANIFEST" "r41..." wid 2.90 ht 0.48 fill 0xe7eef2 at 0,-0.46
OD: box "D  2026-09-13..." wid 2.90 ht 0.26 fill 0xf3f4f3 at 0,OH.s.y - 0.13
OR: box "R  ..." wid 2.90 ht 0.26 fill 0xf3f4f3 at 0,OD.s.y - 0.13
OS1: box "S  A  a17..." wid 2.90 ht 0.26 fill 0xe7eef2 at 0,OR.s.y - 0.13
OS2: box "S  B  b28..." same at 0,OS1.s.y - 0.13
OS3: box "S  C  c36..." same at 0,OS2.s.y - 0.13
OU: box "U  alice" wid 2.90 ht 0.26 fill 0xf3f4f3 at 0,OS3.s.y - 0.13
OZ: box "Z  ...   (final card)" wid 2.90 ht 0.26 fill 0xe7eef2 at 0,OU.s.y - 0.13
OA: box "LEAF" "SUBMANIFEST" "A  ·  a17..." "F run" "Z final" wid 1.18 ht 0.88 fill 0xe7eef2 at -1.24,OZ.s.y - 0.85
OB: box "LEAF" "SUBMANIFEST" "B  ·  b28..." "F run" "Z final" same fill 0xf3f4f3 at 0,OA.y
OC: box "LEAF" "SUBMANIFEST" "C  ·  c36..." "F run" "Z final" same fill 0xe7eef2 at 1.24,OA.y
arrow from (OA.x,OZ.s.y) to OA.n color 0x53616a
arrow from (OB.x,OZ.s.y) to OB.n color 0x53616a
arrow from (OC.x,OZ.s.y) to OC.n color 0x53616a

text "CHECK-IN 2" at 0,OA.s.y - 0.38 color 0x53616a
NH: box "TREE CHECK-IN MANIFEST" "r92...  changed" wid 2.90 ht 0.48 fill 0xf4e8e2 color 0x8b4a32 thickness 0.011 at 0,OA.s.y - 0.85
ND: box "D  2026-09-14..." wid 2.90 ht 0.26 fill 0xf3f4f3 at 0,NH.s.y - 0.13
NR: box "R  ..." wid 2.90 ht 0.26 fill 0xf3f4f3 at 0,ND.s.y - 0.13
NS1: box "S  A   a17..." wid 2.90 ht 0.26 fill 0xe7eef2 at 0,NR.s.y - 0.13
NS2: box "S  B'  e90...   changed" same fill 0xf4e8e2 color 0x8b4a32 thickness 0.011 at 0,NS1.s.y - 0.13
NS3: box "S  C   c36..." same fill 0xe7eef2 color 0x29333a thickness 0.006 at 0,NS2.s.y - 0.13
NU: box "U  alice" wid 2.90 ht 0.26 fill 0xf3f4f3 color 0x29333a thickness 0.006 at 0,NS3.s.y - 0.13
NZ: box "Z  ...   (final card)" wid 2.90 ht 0.26 fill 0xe7eef2 at 0,NU.s.y - 0.13
NA: box "LEAF" "SUBMANIFEST" "A  ·  a17..." "F run" "Z final" wid 1.18 ht 0.88 fill 0xe7eef2 at -1.24,NZ.s.y - 0.85
NB: box "LEAF" "SUBMANIFEST" "B' ·  e90..." "F run" "Z final" same fill 0xf4e8e2 color 0x8b4a32 thickness 0.011 at 0,NA.y
NC: box "LEAF" "SUBMANIFEST" "C  ·  c36..." "F run" "Z final" same fill 0xe7eef2 color 0x29333a thickness 0.006 at 1.24,NA.y
arrow from (NA.x,NZ.s.y) to NA.n color 0x53616a
arrow from (NB.x,NZ.s.y) to NB.n color 0x8b4a32 thickness 0.011
arrow from (NC.x,NZ.s.y) to NC.n color 0x53616a

text "same A hash" at NA.x,NA.s.y - 0.24 color 0x53616a
text "B changed" at NB.x,NB.s.y - 0.24 color 0x8b4a32
text "same C hash" at NC.x,NC.s.y - 0.24 color 0x53616a

The sharing is a property of content addressing, not of any explicit copy-on-write machinery. If two check-ins produce byte-identical sub-manifest bytes, they name the same artifact. No history is rewritten and no reference counting is needed. The check-in manifest remains the root artifact. Its D and Z cards remain required; ordinary context cards such as C, P, R, T, and U remain conditional. Only the file-set representation changes from an F run to an S run.

The S card and the SUBMANIFEST type

The S card changes several existing artifact-format rules. This chapter shows each affected rule and the resulting artifact types.

Grammar and arity

Exactly two arguments. A third token on S is rejected, because the exact-arity enumeration now includes this card. The filename is escape-decoded and must pass the pathname predicate; the hash is raw — never decoded — and must be exactly 40 or 64 case-insensitive hexadecimal characters, matching the artifact-hash validation already applied to other hash-bearing positions.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.0
boxrad = 0.018

# The displayed hash is deliberately abbreviated to keep the anatomy legible.
# On the wire the final token is exactly 40 or 64 hexadecimal characters.
C1: box "S" wid 0.38 ht 0.40 fill 0xe7eef2
C2: box "alpha/lib/parse.c" wid 1.48 ht 0.40 fill 0xf7f8f7 with .w at C1.e
C3: box "7f3a...bb02" wid 1.12 ht 0.40 fill 0xf7f8f7 with .w at C2.e

line from C1.s down 0.30 color 0x53616a
line from C2.s down 0.30 color 0x53616a
line from C3.s down 0.30 color 0x53616a
text "card letter" at C1.x, C1.s.y - 0.48 color 0x29333a
text "decoded filename" at C2.x, C2.s.y - 0.48 color 0x29333a
text "raw hash token" at C3.x, C3.s.y - 0.48 color 0x29333a
text "hash abbreviated here" "wire token: exactly 40 or 64 hex" at C2.x, C2.s.y - 0.82 color 0x53616a
text "filename = least member of the referenced slice" at C2.x, C2.s.y - 1.22 color 0x29333a

Occurrence class and ordering

S joins the repeatable occurrence class, which becomes F, J, M, T, P, Q, S. The intra-run ordering rule requires S cards to be strictly ascending by decoded filename: equal or descending filenames are rejected at parse time, under the canonical comparator that already governs the F run.

Inter-type ordering is unchanged, which places S between R and T. So a manifest's S run precedes its T cards. This matters because it means the ordering amendment is local: no existing card's position in a manifest changes.

What an S card asserts

Each card announces that the referenced artifact is a sub-manifest carrying the contiguous slice of this artifact's file set whose least member is exactly the decoded filename. The slice extends to, and excludes, the next S card's decoded filename. The final card's slice is unbounded above.

Two properties are not checked at parse time, deliberately:

• The hash must resolve to an artifact, and that artifact must be of type SUBMANIFEST.
• The referenced slice must actually begin at the filename the card names.

Both are enforced by the tree walk. Parse-time acceptance validates only token shapes and intra-run ordering. A reference to any type other than SUBMANIFEST must cause the tree walk to fail.

Type determination

SUBMANIFEST is type code 9. It has no card letter that assigns it directly. Instead, one reassignment rule runs after the fallback type determination finalizes a type and before the card-legality checks:

An artifact whose finalized type is MANIFEST and which contains no D card MUST be reclassified as SUBMANIFEST (code 9).

The rule is deliberately narrow. After B, F, Q, R, or S provisionally selects MANIFEST, absence of D reclassifies the artifact as SUBMANIFEST. Other artifact types are unaffected. The legality table then rejects a D-less artifact carrying any card outside F, S, and Z.

The legality table gains a row and revises one:

A signature is not an artifact card and therefore does not appear in this table. PGP or SSH signature armor wraps the complete stored artifact and is removed before its card stream is classified. A tree check-in can therefore be signed exactly like a flat check-in: the signed wrapper surrounds the top-level MANIFEST, while its S references continue to name the child artifacts. The wrapper participates in the manifest's artifact name; the inner Z checksum is still computed over the stripped card stream. A generic reader can unwrap a signed structural artifact, but canonical construction leaves generated SUBMANIFEST nodes unsigned so that independent writers produce the required byte-identical node artifacts.

For SUBMANIFEST only, the content rule requires at least one F or at least one S, never both. It is a semantic check, not expressible in the table's required-set column. Separate MANIFEST rules forbid both B and S, and forbid both F and S.

A leaf sub-manifest is not a delta. Every F card in it must carry a content hash; the hashless deletion form is forbidden. Permission and old-name tokens remain legal exactly as in a baseline manifest, and the tree walk must deliver those tokens to consumers unchanged. Moving the card into a leaf changes where it is stored, not what the file entry means.

The pre-filter exception

The well-formedness pre-filter may reject input whose first card letter falls in N through Z before full parsing. S must be excluded from that fast rejection, because an interior sub-manifest's smallest allowed card letter is S. A pre-filter that rejects on N–Z without the exception rejects every interior node.

The partition function

The partition is part of the check-in identity. Give two conforming writers the same ordered file entries — including every token — the same repository hash policy, and the same pinned $k$. They must build byte-identical sub-manifests and the same manifest S run. Any difference propagates through the referenced artifact hashes and produces a different check-in hash.

The predicate

Define $\operatorname{PH}(\mathit{name})$ as the SHA3-256 digest of the decoded filename — the byte string ordered by the canonical comparator, with no terminator and no transformation. Then:

Why the hashes balance the tree

The partition is deterministic, but its distribution is random-looking. A particular decoded filename always produces the same SHA3-256 digest and therefore the same zbits value. Nothing is sampled again when the repository is rebuilt. Across a varied set of filenames, however, SHA3 output is expected to behave like uniformly distributed bits: the chance that the first bit is zero is one half, the chance that the first two bits are both zero is one quarter, and in general:

This is the source of the tree's statistical balance. A smaller $k$ makes qualifying hashes common, producing more, smaller leaves and more S references. A larger $k$ makes them rarer, producing fewer, larger leaves and usually fewer levels. At each higher level the threshold gains another $k$ zero bits, so natural boundaries thin out by roughly another factor of $2^k$. The shape is balanced in expectation, not perfectly balanced for every filename set.

The group limits make that probabilistic scheme safe. MIN suppresses a natural boundary that arrives too soon, while MAX forces a boundary when the hashes produce an unusually long run. Thus SHA3 supplies stable, content-derived randomness; $k$ sets its target scale; and MIN/MAX constrain the outliers.

Every boundary test compares zbits against a multiple of a single positive integer $k$, the partition parameter. A leaf boundary requires $\operatorname{zbits}(\mathit{name}) \ge k$. A boundary between height-$h$ nodes requires $\operatorname{zbits}(\mathit{key}) \ge (h+1)k$.

For $k \le 2$, MIN may be 0 or 1, so MIN alone cannot prevent single-child spines; the non-reducing-scan stop and the height cap provide the remaining bounds.

The digest algorithm, the bit granularity, and the input bytes are fixed and must not be configurable or negotiable per artifact. The predicate reads the filename only: content hash, permission, and old-name never influence the partition. Thus editing a file's content changes one leaf's contents without moving partition boundaries; the hashes of that leaf's ancestors and the check-in manifest change in consequence.

Group limits

Two limits count members — file entries at height 0, child nodes above — never bytes:

MAX forces a split when a group would otherwise grow without bound; this is the content-defined-chunking corrective inherited from FastCDC, and it bounds the pathological case where no filename's digest qualifies. MIN suppresses tiny groups and usually prevents one-child spines. For k values 1 and 2, however, MIN is 0 or 1 and cannot suppress a singleton; the non-reducing-scan stop and height cap provide the bound instead.

The scan

Construction begins from the check-in's file entries $e_1 \lt \cdots \lt e_N$ in canonical filename order — the same entries a flat baseline manifest would emit as F cards after the reserved-name exclusions.

Step 1 scans $e_1,\ldots,e_N$ once, left to right, into height-0 groups. $e_1$ starts the first group. Each subsequent entry $e_i$ starts a new group if and only if the open group holds MAX members, or $\operatorname{zbits}(\mathit{name}(e_i)) \ge k$ and the open group holds at least MIN members.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.05
boxrad = 0.018

# Separate schematic scans show the two alternatives without fake limits.
text "natural boundary" at 0,0 color 0x29333a
G1: box "open group" "size >= MIN" wid 1.28 ht 0.56 fill 0xe7eef2 at 0,-0.52
N1: box "next name" "zbits(name) >= k" wid 1.45 ht 0.56 fill 0xf7f8f7 at G1.x + 1.72, G1.y
arrow from G1.e to N1.w color 0x53616a
O1: box "start new group" wid 1.20 ht 0.40 fill 0xe7eef2 at N1.x, N1.s.y - 0.52
arrow from N1.s to O1.n color 0x53616a

text "forced boundary" at 0,O1.s.y - 0.16 color 0x29333a
G2: box "open group" "size = MAX" wid 1.28 ht 0.56 fill 0xf4e8e2 color 0x8b4a32 at 0,O1.s.y - 0.58
N2: box "next name" "predicate may fail" wid 1.45 ht 0.56 fill 0xf7f8f7 at G2.x + 1.72, G2.y
arrow from G2.e to N2.w color 0x8b4a32
O2: box "start new group" wid 1.20 ht 0.40 fill 0xf4e8e2 color 0x8b4a32 at N2.x, N2.s.y - 0.52
arrow from N2.s to O2.n color 0x8b4a32

text "MIN and MAX come from k" "the split causes are distinct" at (G2.x + N2.x)/2, O2.s.y - 0.30 color 0x53616a

When the scan completes, one merge rule applies: if the final group holds fewer than MIN members, a preceding group exists, and the two together hold at most MAX members, the final group is merged into the group preceding it. Without this, a trailing fragment would become its own undersized leaf.

Step 2 checks for degenerate outcomes. If the scan yields fewer than two groups, the writer must emit a flat manifest and no sub-manifests. The case N = 0 — an empty check-in, which Fossil permits — yields zero groups and a flat manifest carrying no F cards and no S cards.

If the height-0 scan yields at least two groups, each group becomes a leaf sub-manifest carrying exactly that group's ordered F cards and a final Z card. Subject only to the emission gate, the avoid-submanifests pragma, and the out-of-range-pin fallback, an enabled writer must emit this canonical tree. Choosing a flat manifest instead is nonconforming.

The key of a node is the decoded filename of the first file entry in its subtree.

Recursion and termination

For $h=1,2,\ldots$, let $n_1,\ldots,n_m$ be the height-$h-1$ nodes in key order. Partition them by the same scan, including the final-group merge, with the boundary test $\operatorname{zbits}(\mathit{key}(n_i)) \ge (h+1)k$ and with MIN and MAX counting nodes.

The RFC's four-file Appendix A vector makes that rule concrete. With $k=3$, $\mathrm{MIN}=2$, and $\mathrm{MAX}=32$, only src/node11.c qualifies at height 0, after two entries are already open. The result is exactly two leaves.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.08
boxrad = 0.018

# RFC Appendix A.1-A.3, rendered top-to-bottom. Hashes are explicitly
# abbreviated only for this illustration.
text "APPENDIX A VECTOR   k=3   MIN=2   MAX=32" at 0,0 color 0x53616a
text "hashes abbreviated only in this figure" at 0,-0.22 color 0x53616a
MH: box "TREE CHECK-IN MANIFEST" "95fb...c509" "S-RUN EXCERPT; OTHER CARDS OMITTED" wid 2.82 ht 0.66 fill 0xe7eef2 at 0,-0.66
MS1: box "S README.md -> aa9d...0acf" wid 2.82 ht 0.34 fill 0xe7eef2 at MH.x,MH.s.y - 0.17
MS2: box "S src/node11.c -> a50a...f52ea" same at MH.x,MS1.s.y - 0.17

L1H: box "LEAF SUBMANIFEST" "aa9d...0acf" "key README.md" wid 1.78 ht 0.62 fill 0xf3f4f3 at -0.96,MS2.s.y - 0.88
L1A: box "F README.md   zbits 1" wid 1.78 ht 0.34 fill 0xf7f8f7 at L1H.x,L1H.s.y - 0.17
L1B: box "F src/main.c  zbits 0" same at L1H.x,L1A.s.y - 0.17
L1Z: box "Z  bdbb...53f1" same fill 0xe7eef2 at L1H.x,L1B.s.y - 0.17
L2H: box "LEAF SUBMANIFEST" "a50a...f52ea" "key src/node11.c" wid 1.78 ht 0.62 fill 0xf3f4f3 at 0.96,L1H.y
L2A: box "F src/node11.c  zbits 8" wid 1.78 ht 0.34 fill 0xf7f8f7 at L2H.x,L2H.s.y - 0.17
L2B: box "F src/util.c    zbits 0" same at L2H.x,L2A.s.y - 0.17
L2Z: box "Z  87c5...4a6e" same fill 0xe7eef2 at L2H.x,L2B.s.y - 0.17
arrow from MS1.sw to L1H.n color 0x53616a
arrow from MS2.se to L2H.n color 0x53616a

T: box "height 1: src/node11.c has zbits 8" "8 >= 2k = 6" "but open group has 1 node < MIN = 2" wid 3.08 ht 0.72 fill 0xf4e8e2 color 0x8b4a32 at 0,L1Z.s.y - 0.68
line from L1Z.s to T.nw color 0x53616a
line from L2Z.s to T.ne color 0x53616a
R: box "split suppressed -> one group" "manifest names both leaves directly" "no interior node" wid 3.08 ht 0.72 fill 0xe7eef2 at 0,T.s.y - 0.65
arrow from T.s to R.n color 0x8b4a32

This small vector stops with the manifest's two S cards because the height-1 scan yields one group. Larger file sets can yield multiple groups at that level; those groups become interior, S-only sub-manifests, and the same scan repeats upward.

What changing k tends to change

The predicate qualifies a filename with probability about $2^{-k}$, so the unconstrained expected group size is about $2^k$ members. MIN and MAX remain corrective limits, not a promise that every group falls above a universal lower bound: final merging, degenerate scans, and the actual filename hashes still determine the concrete tree.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.10
boxrad = 0.018

text "EXPECTED SCALE AS k CHANGES" at 0,0 color 0x53616a

K1: box "k=1" "expected ~2" "MIN=0  MAX=8" wid 1.12 ht 0.66 fill 0xf7f8f7 at -1.18,-0.58
A1: box "leaf group" wid 0.64 ht 0.34 fill 0xe7eef2 at -0.18,K1.y + 0.16
A2: box "leaf group" same at A1.x + 0.70,A1.y
A3: box "leaf group" same at A2.x + 0.70,A2.y
A4: box "leaf group" same at A3.x + 0.70,A3.y
text "more small leaves / references" "often more levels" at 0.74,K1.y - 0.25 color 0x53616a

K3: box "k=3" "expected ~8" "MIN=2  MAX=32" wid 1.12 ht 0.66 fill 0xf7f8f7 at K1.x,K1.y - 1.00
B1: box "leaf group" wid 0.78 ht 0.34 fill 0xe7eef2 at 0.02,K3.y + 0.16
B2: box "leaf group" same at B1.x + 0.84,B1.y
B3: box "leaf group" same at B2.x + 0.84,B2.y
text "middle scale" at 0.80,K3.y - 0.25 color 0x53616a

K5: box "k=5" "expected ~32" "MIN=8  MAX=128" wid 1.12 ht 0.66 fill 0xf7f8f7 at K1.x,K3.y - 1.00
C1: box "larger expected leaf group" wid 2.04 ht 0.34 fill 0xe7eef2 at 0.80,K5.y + 0.16
text "fewer larger leaves / references" "often fewer levels" at 0.80,K5.y - 0.25 color 0x53616a

N: box "EXPECTATION, NOT A GUARANTEE" "for a fixed filename set, results may be" "non-monotonic as k changes" wid 3.18 ht 0.74 fill 0xf4e8e2 color 0x8b4a32 at 0,K5.s.y - 0.68
T: box "lookup tradeoff" "fewer levels  <->  more bytes in the chosen leaf" wid 3.18 ht 0.58 fill 0xe7eef2 at 0,N.s.y - 0.55

That is the lookup tradeoff: a higher $k$ often means fewer levels to traverse but more bytes in the final leaf, while a lower $k$ often means smaller leaves reached through more references. This tendency motivates parameter selection; it does not alter reader correctness.

Two stop conditions prevent degenerate upper levels:

• If the scan, after the merge, yields a single group, the writer disregards it and emits one S card per height-(h-1) node in the manifest itself. Construction terminates.
• If the scan yields as many groups as it was given nodes — a non-reducing scan that would make every node a one-child parent — the same thing happens.

Otherwise each group becomes an interior sub-manifest carrying one S card per member node, in key order, and a Z card, and the process repeats at height $h+1$.

A third stop condition is a hard cap: construction must not create a node of height 16. If the scan at $h=16$ yields more than one group after the merge, the writer emits one S card per height-15 node in the manifest. This keeps every canonical tree within the reader-side depth bound.

Canonical emission

Rule. Each S filename is the canonical encoding of the child's key. Its hash is the child's artifact name under the repository hash policy.

Example. Canonical encoding uses minimal escaping: SP becomes \s, LF becomes \n, and backslash becomes \\. TAB, CR, VT, FF, NUL, and every other octet are emitted raw; escaping any of them is non-canonical.

Consequence. The same encoding rule applies to S-card filenames and to the filename and old-name tokens of leaf F cards. Decoding restores the original decoded name exactly.

Sub-manifest artifacts are stored before the manifest that references them. Content-addressed storage deduplicates any node identical to one already stored. A writer may reuse a subtree of a parent check-in's tree provided the resulting artifacts are byte-identical to those of the full construction — reuse is position-dependent under the group limits.

Parameter selection

Selection is advisory, not canonical. It begins from the repository's current check-in — the parent of the next commit — resolved once. If there is no current check-in, automatic selection has no result. For each $k \in \{1,\ldots,12\}$, the implementation performs the full construction under the SHA3 hash policy. A value is a candidate only when its height-0 scan yields at least two groups; in other words, only when it actually produces a tree.

For each candidate, let $L_i$ be the complete serialized byte length of leaf artifact $i$, including every card and terminating LF. Let $S$ be the sum of complete serialized byte lengths, including LF, of every S card that names a leaf. Then:

An implementation should choose the lowest-cost candidate and must choose the lowest $k$ among candidates with equal minimum cost. If no value is a candidate, automatic selection has no result, and an implementation must not silently substitute a value. An operator may supply any value from 1 to 12 inclusive; any pinned value yields conforming artifacts.

The analyzed optimum tracks roughly half the base-2 logarithm of the file count. The cost function's second term is a size-weighted mean leaf size, so oversized leaves weigh more heavily in the choice. It is an optimization objective, not a safety bound: the hard MAX group limit and the depth cap are what bound tree shape and lookup work.

Re-tuning

The pinned parameter may be changed at any time; a re-tune is forward-only. The first commit after the change is constructed under the new value. No history is rewritten, and a reader must not behave differently across the change.

Sharing across a re-tune survives exactly for nodes whose full recursive construction remains byte-identical. Repositories pinned to different values exchange artifacts correctly — differing values can reduce sharing but do not prevent it. So propagation of $k$ need be reliable only for cost, not for correctness.

Because a re-tune can reduce sharing and increase the artifact bytes introduced by the next commit, it must be manual and operator-initiated. Automatic re-tuning requires the submanifest-auto-retune project-scope setting — default false, and it must not be inferred from repository state. When enabled, the writer retains the current value unless the candidate is distinct and:

Consequences

Between the limits, group sizes follow the geometric distribution of the boundary test with mean $2^k$ members. Tree depth grows logarithmically, base $2^k$.

An insertion or deletion can cascade — a boundary decision depends on the group preceding it — but the next natural boundary preceded by at least MIN members re-synchronizes the scan. Editing a file's content changes one leaf's contents without changing the partition; that leaf's ancestor hashes and the check-in manifest necessarily change as the new hash propagates to the root.

Measured on eight Fossil-hosted repositories (28 to 2,215 files, analyzed $2 \le k \le 5$), expected bytes read to resolve one file dropped by a factor of 2.2 to 19.9 versus flat F card bytes. Exhaustive insertion and deletion sweeps changed 0.96 to 1.38 leaves on average, worst observed 3. There is no rebalance operation: every boundary is decided by the one-pass scan at construction. The construction is that of Merkle Search Trees with the standard content-defined-chunking group-limit corrective.

Tree reconstruction

A top-level MANIFEST is a tree manifest if and only if it contains an S card. Every referenced node must be resolvable; each is parsed when the walk encounters it. Consumers that need the complete file set must use that walk, which also enforces canonical ordering across every S and F run.

The walk

Enumeration is a depth-first, in-order walk over the S references, maintaining a stack of frames — one per artifact being read, each with a cursor over that artifact's S run — and a running value LAST, the decoded filename most recently emitted.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.02
boxrad = 0.018

# A sequential trace of the depth-first walk. Each row resolves one S card,
# validates the leaf, emits its sorted F run, and advances LAST before the
# next S card is visited. Artifact hashes are deliberately abbreviated.
text "DEPTH-FIRST WALK   one S card at a time" at 0,0.90 color 0x53616a
text "TREE CHECK-IN" "MANIFEST S card" at -1.40,0.68 color 0x53616a
text "resolved artifact" at 0,0.63 color 0x53616a
text "validation state" at 1.40,0.63 color 0x53616a

S1: box "1   S aardvark" "a17..." wid 1.08 ht 0.56 fill 0xe7eef2 at -1.40,0
L1: box "LEAF SUBMANIFEST" "F aardvark" "F bee" "F cat" "Z final" wid 1.34 ht 1.02 fill 0xf7f8f7 at 0,S1.y
V1: box "type valid" "F run sorted" "first F = S key" "emit 3; LAST = cat" wid 1.12 ht 0.84 fill 0xe7eef2 at 1.40,S1.y
arrow from S1.e to L1.w color 0x53616a
arrow from L1.e to V1.w color 0x53616a

text "cross-leaf check:  cat < dog  (next S key)" at 0,L1.s.y - 0.22 color 0x3f7550

S2: box "2   S dog" "b28..." wid 1.08 ht 0.56 fill 0xe7eef2 at S1.x,S1.y - 1.38
L2: box "LEAF SUBMANIFEST" "F dog" "F elephant" "F elk" "Z final" wid 1.34 ht 1.02 fill 0xf7f8f7 at L1.x,S2.y
V2: box "type valid" "F run sorted" "first F = S key" "emit 3; LAST = elk" wid 1.12 ht 0.84 fill 0xe7eef2 at V1.x,S2.y
arrow from S2.e to L2.w color 0x53616a
arrow from L2.e to V2.w color 0x53616a

text "cross-leaf check:  elk < goose  (next S key)" at 0,L2.s.y - 0.22 color 0x3f7550

S3: box "3   S goose" "c36..." wid 1.08 ht 0.56 fill 0xe7eef2 at S2.x,S2.y - 1.38
L3: box "LEAF SUBMANIFEST" "F goose" "Z final" wid 1.34 ht 1.02 fill 0xf7f8f7 at L2.x,S3.y
V3: box "type valid" "F run sorted" "first F = S key" "emit 1; LAST = goose" wid 1.12 ht 0.84 fill 0xe7eef2 at V2.x,S3.y
arrow from S3.e to L3.w color 0x53616a
arrow from L3.e to V3.w color 0x53616a

O: box "CONCATENATED F-CARD ENUMERATION" "aardvark < bee < cat < dog < elephant < elk < goose" wid 3.68 ht 0.64 fill 0xe7eef2 at 0,L3.s.y - 0.55
arrow from V3.s to O.ne color 0x53616a
text "LAST after each leaf:  cat  ->  elk  ->  goose" at 0,O.s.y - 0.25 color 0x53616a

Step by step:

1. Initialize the stack with one frame over the manifest's S run, and LAST as the absent value, which sorts before any present value.
2. If the top frame's cursor is exhausted, pop the frame. When the popped frame is the manifest's, enumeration terminates.
3. Otherwise read the cursor's S card. Its decoded filename must compare strictly greater than LAST. Resolve its hash.
4. If the resolved sub-manifest is a leaf, emit its F cards in order. The first emitted decoded filename must be byte-identical to the S card's decoded filename, and every emitted filename must compare strictly greater than LAST, updating LAST as it is emitted. Then advance the referencing frame's cursor.
5. If the resolved sub-manifest is interior, its first S card's decoded filename must be byte-identical to the referencing card's decoded filename. Advance the referencing frame's cursor, push a frame over the resolved sub-manifest's S run.

Four violations at step 3 must cause the enumeration to fail:

• The referenced artifact is absent.
• The referenced artifact is not of type SUBMANIFEST.
• The referenced artifact is the artifact of any frame currently on the stack.
• The referenced artifact's depth exceeds 16 — counting sub-manifest artifacts on the reference path from the manifest down to and including the resolved artifact, leaves included.

That third check is the cycle rejection, and together with the depth bound it is the entire defense against a reference cycle in attacker-suppliable artifacts. Downstream consumers — the R card computation, file-difference walks, checkout — must not distinguish a walked enumeration from a flat or delta one.

Resolution discipline

S card hashes are resolved lazily, on first need, under the same two disciplines as the B card baseline:

• Hard failure. The enumeration entry points — beginning the tree walk, and single-name lookup — must treat an unresolvable sub-manifest as a fatal error.
• Soft failure. Consumers that can defer, such as post-parse relationship processing, may record the gap keyed by the dependent artifact and the missing hash, inserted idempotently, report failure to the caller, and retry when the sub-manifest arrives.

An implementation must not present a partially resolved tree as a complete file set. There is no degraded partial enumeration analogous to the delta-only degradation the artifact specification cautions against.

Single-name lookup

Resolving one filename against a tree manifest never requires the full walk. Binary-search the S run for the last card whose decoded filename compares less than or equal to the target; if no such card exists the name is absent. Otherwise resolve that card's sub-manifest and repeat — the S run of an interior node, the F run of a leaf — until a leaf either contains the name or proves it absent.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.0
boxrad = 0.018

text "SINGLE-NAME LOOKUP" at 0,0.48 color 0x53616a
Q: box "target filename  ·  elephant" wid 1.62 ht 0.38 fill 0xf7f8f7

text "TREE CHECK-IN" "MANIFEST S cards" at -1.18,Q.s.y - 0.30 color 0x53616a
S2: box "S  dog  b28..." wid 0.92 ht 0.38 fill 0xf4e8e2 color 0x8b4a32 thickness 0.011 at 0,Q.s.y - 0.66
S1: box "S  aardvark  a17..." wid 1.18 ht 0.38 fill 0xf7f8f7 with .e at S2.w
S3: box "S  goose  c36..." wid 1.08 ht 0.38 fill 0xf7f8f7 color 0x29333a thickness 0.006 with .w at S2.e
arrow from Q.s to S2.n color 0x8b4a32 thickness 0.011

LH: box "LEAF SUBMANIFEST" "b28..." wid 1.62 ht 0.54 fill 0xe7eef2 at 0,S2.s.y - 0.62
arrow from S2.s to LH.n color 0x8b4a32 thickness 0.011
text "last S key <= target" at 1.22,(S2.s.y + LH.n.y)/2 color 0x8b4a32

F1: box "F  dog" wid 1.62 ht 0.30 fill 0xf7f8f7 at 0,LH.s.y - 0.15
F2: box "F  elephant" same fill 0xf4e8e2 color 0x8b4a32 thickness 0.011 at 0,F1.s.y - 0.15
F3: box "F  elk" same fill 0xf7f8f7 color 0x29333a thickness 0.006 at 0,F2.s.y - 0.15
FZ: box "Z  ...  (final card)" same fill 0xe7eef2 at 0,F3.s.y - 0.15

text "FOUND  ·  one SUBMANIFEST read at this level" at 0,FZ.s.y - 0.24 color 0x3f7550
text "one referenced artifact per level  ·  hard depth limit 16" at 0,FZ.s.y - 0.50 color 0x53616a

The depth bound must be enforced here too: a lookup that would resolve a sub-manifest at depth greater than 16 must fail hard. In particular, without this check a crafted deep spine turns an unauthenticated lookup endpoint into unbounded sequential artifact reads. At most one sub-manifest per tree level, hence at most 16 per lookup, is read.

Composition with delta manifests

A tree manifest carries no B card, so it is a baseline manifest under the existing predicate. Therefore a delta manifest may name a tree manifest in its B card. Full enumeration composes the two walks: the baseline cursor of the merge-walk becomes the enumeration above, and the merge-walk itself is unchanged.

One writer-side precaution remains even after tree emission is turned off: a writer should not create a new delta manifest whose baseline is a tree manifest. Existing immutable deltas remain readable through the composed walk; the recommendation prevents disabled writers from introducing new dependencies on tree baselines.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.0
boxrad = 0.018

T: box "TREE CHECK-IN MANIFEST" "baseline file set" wid 1.62 ht 0.52 fill 0xe7eef2
C: box "in-order cursor" wid 1.18 ht 0.40 fill 0xf7f8f7 at T.x + 1.68, T.y
arrow from T.e to C.w color 0x53616a

D: box "DELTA MANIFEST" "delta F run" wid 1.62 ht 0.52 fill 0xf7f8f7 at T.x, T.s.y - 0.76
M: box "EXISTING DELTA" "merge walk" wid 1.28 ht 0.56 fill 0xe7eef2 at C.x, D.y
arrow from C.s to M.n color 0x53616a "baseline cursor"
arrow from D.e to M.w color 0x53616a
text "delta entries" at D.x, D.s.y - 0.18 color 0x53616a

E: box "effective sorted file set" wid 1.88 ht 0.42 fill 0xf3f4f3 at M.x, M.s.y - 0.62
arrow from M.s to E.n color 0x53616a
text "tree enumeration supplies the cursor" "delta precedence is unchanged" at (T.x + C.x)/2, E.s.y - 0.36 color 0x29333a

Single-name lookup follows the artifact specification's delta lookup procedure with the section 5.3 lookup as the baseline-search step. One writer-side constraint applies: while enable-submanifests is in force, a writer must not create a delta manifest at all. The setting overrides the seen-delta-manifest latch.

Checksums still cover the same file set

Indirection does not weaken the check-in checksum. A tree manifest's R card is computed over the complete walk enumeration — tracked files in canonical filename order — and then uses Fossil's existing MD5 aggregate algorithm byte for byte. A delta manifest whose baseline is a tree obtains the baseline side of that calculation from the same walk.

There are two useful verification views. Given a checkout, a verifier can recompute R from the checked-out files and compare it with the root manifest without traversing the sub-manifest tree. A repository verifier can instead walk the tree and check each file hash. That walk is simpler than delta expansion: leaves are contiguous sorted slices, so their F runs concatenate in S-card order; there is no two-cursor merge and no deletion-marker precedence.

Every sub-manifest's Z card detects accidental corruption in that node's stripped card stream. The artifact's content hash — not Z — cryptographically names and binds its complete stored bytes.

Memoization

Implementations should cache per-sub-manifest enumeration results keyed by artifact name. Because unchanged subtrees are shared by hash across check-ins, a memoizing rebuild parses each distinct sub-manifest once — converting rebuild cost from the sum of full file-set sizes to the sum of distinct-artifact sizes.

Sync, phantoms, and the flag day

Sub-manifests are ordinary artifacts for every discovery and transfer purpose: announced by igot, requested by gimme, carried by file and cfile, and included in cluster artifacts through M cards. No wire-format change exists. The requirements are about state and sequencing.

Phantoms and convergence

Storing a structural artifact carrying S cards whose referenced sub-manifests are absent must create a phantom for each missing reference, with the referencing artifact's privacy classification — exactly as an absent delta source creates a source phantom.

The subtlety is timing. An interior sub-manifest's own S references become known only when it arrives. So a tree of depth $d$ converges in at most $d+1$ pull rounds, not one, because each round reveals one more level of the frontier.

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.00
boxrad = 0.018

# A concrete depth-2 frontier expansion. All hashes are abbreviated.
# Dashed rust boxes are newly created phantoms; the green box is closure.
text "PHANTOM FRONTIER EXPANSION   depth-2 example" at 0,0.90 color 0x53616a
text "pull round" at -1.52,0.62 color 0x53616a
text "store, dephantomize, then parse" at 0,0.62 color 0x53616a
text "new S frontier" at 1.52,0.62 color 0x53616a

R1: box "ROUND 1" "receive root" wid 1.02 ht 0.62 fill 0xe7eef2 at -1.52,0
A1: box "TREE CHECK-IN MANIFEST" "a0..." "S aardvark  b1..." "S goose     c2..." wid 1.66 ht 0.92 fill 0xf7f8f7 at 0,R1.y
F1: box "create phantoms" "{ b1..., c2... }" "privacy copied" wid 1.20 ht 0.76 fill 0xffffff color 0x8b4a32 dashed at 1.52,R1.y
arrow from R1.e to A1.w color 0x53616a
arrow from A1.e to F1.w color 0x8b4a32

text "next pull: gimme b1... and c2... together" at 0,A1.s.y - 0.24 color 0x53616a

R2: box "ROUND 2" "receive frontier" wid 1.02 ht 0.62 fill 0xe7eef2 at R1.x,R1.y - 1.30
A2: box "INTERIOR SUBMANIFESTS" "b1...  c2..." "parse both S runs" wid 1.66 ht 0.84 fill 0xf7f8f7 at A1.x,R2.y
F2: box "create phantoms" "{ d3..., e4..., f5... }" "privacy copied" wid 1.20 ht 0.76 fill 0xffffff color 0x8b4a32 dashed at F1.x,R2.y
arrow from R2.e to A2.w color 0x53616a
arrow from A2.e to F2.w color 0x8b4a32

text "next pull: gimme the whole leaf frontier" at 0,A2.s.y - 0.24 color 0x53616a

R3: box "ROUND 3" "= d + 1 here" wid 1.02 ht 0.62 fill 0xe7eef2 at R2.x,R2.y - 1.30
A3: box "LEAF SUBMANIFESTS" "d3...  e4...  f5..." "F runs; no S cards" wid 1.66 ht 0.84 fill 0xf7f8f7 at A2.x,R3.y
F3: box "S frontier empty" "tree closure present" "retry dependents" wid 1.20 ht 0.76 fill 0xe8f1ea color 0x3f7550 at F2.x,R3.y
arrow from R3.e to A3.w color 0x53616a
arrow from A3.e to F3.w color 0x3f7550

G: box "GENERAL BOUND" "manifest round + d sub-manifest levels = at most d + 1 rounds" wid 3.82 ht 0.68 fill 0xe7eef2 at 0,A3.s.y - 0.60
arrow from F3.s to G.ne color 0x53616a
text "already-present artifacts can make it fewer; width does not add rounds" at 0,G.s.y - 0.25 color 0x53616a

A frontier may contain many artifacts and the next pull may request them together, so the bound is controlled by tree depth, not tree width. “Tree closure present” means every reachable SUBMANIFEST is available; file-content blobs named by leaf F cards have their own ordinary phantom state.

Dephantomization of a sub-manifest must retry the deferred processing of every recorded dependent. This connects to a specific implementation trap — see chapter 6.

Crosslink behavior

Crosslinking a tree manifest requires its full enumeration wherever crosslinking a flat manifest requires the F card list. When a referenced sub-manifest is absent, the deferred-processing discipline for a missing delta baseline applies unchanged: record the gap keyed by dependent and missing hash, idempotently, and retry on arrival.

A SUBMANIFEST artifact has no direct crosslink effect. It carries no state-bearing card and contributes to repository state only through the manifests whose trees contain it. Its arrival triggers retry of recorded dependents and nothing else.

The peer-version gate

When the remote peer's announced version predates sub-manifest support and a local artifact is of type SUBMANIFEST, or is a manifest carrying an S card, the sender must refuse to transmit that artifact — with an explicit error, never by converting, downgrading, or silently omitting it. The rule parallels the SHA3-256 peer-version gate.

A server whose enable-submanifests setting is false sends the pragma avoid-submanifests, in the same reply positions as avoid-delta-manifests. A client receiving it must not emit S cards in commits created during the session.

Why there is no fallback

Exactly 22 card letters are defined and the alphabet is closed. A card letter outside A–Z must be rejected; an in-range but undefined letter (O, V, X, Y) must be rejected. There is no forward-compatibility acceptance path for unknown card letters.

So a sub-manifest or tree manifest is unreadable — rejected as structural, held as opaque content — by every reader conforming to the published artifact specification. Such a reader's check-in graph is visibly truncated, never silently divergent. That property is deliberate: a truncated graph is a loud failure, whereas a silently wrong file set would be a data-integrity event.

The settings travel as one decision

Rule. submanifest-k, enable-submanifests, and submanifest-auto-retune travel together in the scalar /project configuration group. A supporting client with Read permission requests that group during pull and, after clone deferral, in the first authorized request. A clone that cannot retrieve the complete authorized group must leave emission disabled.

Transaction. A receiver stages all three entries through the end of the request or reply, then validates and commits them atomically, independent of card order. A supporting server recognizes all three names; pushing them requires Admin permission.

Failure example. An invalid value, or a final state that enables emission without $k \in \{1,\ldots,12\}$ rejects this three-setting transaction and preserves its prior values. Other valid /project settings may still apply.

Consequence. A writer reads k from a successfully stored repository pin and never derives a fresh value per commit. If corrupted or hand-edited local state contains an out-of-range pin, it emits a flat manifest and should warn the operator; it must not substitute another parameter.

Three gates

scale = 0.82
fill = 0xffffff
color = 0x29333a
thickness = 0.006
fontscale = 1.06
boxrad = 0.018

# Three independent compact panels. Each decision has its own input and
# outcomes; no artifact is implied to flow through all three.
text "1  EMISSION — BEFORE CREATION" at 0,0 color 0x53616a
G1: box "enabled and no avoid?" wid 1.55 ht 0.42 fill 0xe7eef2 at 0,-0.44
Y1: box "yes: tree permitted" wid 1.38 ht 0.42 fill 0xe7eef2 at G1.x - 0.82, G1.s.y - 0.58
N1: box "no: flat or refuse enable" wid 1.55 ht 0.42 fill 0xf4e8e2 color 0x8b4a32 at G1.x + 0.82, Y1.y
arrow from G1.sw to Y1.n color 0x53616a
arrow from G1.se to N1.n color 0x8b4a32

text "2  TRANSFER — EXISTING IMMUTABLE S ARTIFACT" at 0,Y1.s.y - 0.38 color 0x53616a
G2: box "peer supports S?" wid 1.55 ht 0.42 fill 0xe7eef2 at 0,Y1.s.y - 0.75
Y2: box "yes: send unchanged" wid 1.38 ht 0.42 fill 0xe7eef2 at G2.x - 0.82, G2.s.y - 0.58
N2: box "no: refuse; never flatten" wid 1.55 ht 0.42 fill 0xf4e8e2 color 0x8b4a32 at G2.x + 0.82, Y2.y
arrow from G2.sw to Y2.n color 0x53616a
arrow from G2.se to N2.n color 0x8b4a32

text "3  PROVISIONAL REPOSITORY-USE OVERLAY" at 0,Y2.s.y - 0.38 color 0x53616a
G3: box "disposable test fixture?" wid 1.55 ht 0.42 fill 0xe7eef2 at 0,Y2.s.y - 0.75
Y3: box "yes: test use only" wid 1.38 ht 0.42 fill 0xe7eef2 at G3.x - 0.82, G3.s.y - 0.58
N3: box "no: do not propagate" wid 1.55 ht 0.42 fill 0xf4e8e2 color 0x8b4a32 at G3.x + 0.82, Y3.y
arrow from G3.sw to Y3.n color 0x53616a
arrow from G3.se to N3.n color 0x8b4a32

Emission. A writer must not emit an S card unless enable-submanifests — a project-scope boolean, default false — is true, and must not emit one in a commit against a server from which it received the avoid-submanifests pragma in the current sync session.

Rule. For a local request to turn emission on, an existing valid pin $k \in \{1,\ldots,12\}$ wins. If the request also supplies a different explicit value, the writer rejects the request atomically and directs the operator to re-tuning.

Example. With no valid pin, an explicit value from 1 through 12 may be stored. An explicit out-of-range value is rejected; it is not treated as omitted and does not trigger automatic selection. If the value is omitted, the writer runs automatic selection.

Consequence. If automatic selection has no result, the writer refuses the request atomically: enable-submanifests and submanifest-k retain their prior values, and no S card is emitted. Emission must never be enabled without a valid pin.

Disabling the setting stops the emission of new tree manifests; it cannot retire existing ones, because artifact names are immutable. When the first S-bearing artifact is stored, an implementation should record a local seen-submanifest indicator so operational surfaces can warn before the setting is enabled or a pre-S client is granted access.

Provisional phase. Until this specification folds into a -01 revision of the artifact format and conforming readers ship in released versions of the implementations that serve a repository's users, every repository containing an S-bearing artifact is a disposable test fixture:

• It must not be synchronized with, cloned to, or otherwise propagated into any repository that is not itself a disposable test fixture.
• The setting must not be enabled on any repository that any pre-S implementation serves.
• Implementers must assume any element of the specification may still change, and that a change invalidates every existing S-bearing artifact without migration.

The four structural traps

These requirements constrain implementation structure rather than artifact bytes. Each prevents a specific failure mode identified in structural audits of the four extant Fossil implementations.

1. Type determination without a positive signal

SUBMANIFEST must be determined through the reassignment rule — a post-pass keyed on the absence of a D card — and an implementation must not commit to a type while cards are still being read. Required-card logic that assumes every type requires a D card must treat SUBMANIFEST, like CLUSTER, as an explicit exception.

The failure mode: a parser that decides type incrementally will classify a sub-manifest as a manifest on the strength of its first F or S card, then hold a type that the legality table will later contradict.

2. Sweep termination

A SUBMANIFEST artifact never produces a derived row of its own. An implementation that discovers crosslink work by negative space — selecting every structural artifact for which no derived row exists — must exclude sub-manifest artifacts from the sweep, or record an explicit completion marker for them.

The failure mode is a livelock: the sweep re-selects and re-parses the same sub-manifests on every pass, forever, because they never acquire the row the selection predicate is looking for.

3. The enumeration seam

Every consumer of a check-in's file set — checkout, diff, crosslink, rebuild, annotate, commit — must obtain it through one shared enumeration interface implementing the tree walk and its delta composition. Implementations must route every bypass site through the interface before enabling the reader, and should enforce the seam with an assertion or type distinction.

A bypass site is any place that iterates a raw file-card array. Such a site either fails on tree manifests or silently diverges from the seam — and the second is worse, because it produces a plausible but incomplete file set.

4. Dephantomization of S references

Deferred-processing cascades typically know two edge kinds: delta-manifest baselines and content delta sources. The S reference is a third.

An implementation must record, when a tree manifest's processing is deferred, enough to rediscover that manifest when the missing sub-manifest arrives, and its dephantomization path must traverse that record. A cascade that walks only the two older edge kinds leaves such a check-in permanently unprocessed.

Security notes

The tree walk recurses through attacker-suppliable artifacts. The depth bound and the repeated-artifact check together reject reference cycles and bound the stack. Implementations should bound total sub-manifests visited per enumeration and must bound it when serving unauthenticated requests — in particular, the depth bound must be enforced in single-name lookup.

A phantom sub-manifest leaves a tree unresolvable, exactly as a phantom baseline leaves a delta manifest unresolvable. Shunning a sub-manifest renders every check-in whose tree contains it unenumerable — a strictly wider blast radius than shunning one file's content, so operators should shun sub-manifests only with the same caution as baseline manifests.

The partition predicate is not an integrity mechanism. Integrity binds through S and F card hashes; SHA3-256's role in the partition is only to choose boundaries. A committer who controls filenames can shape the tree within the group limits: all-non-qualifying names partition into MAX-member groups once the file count exceeds MAX, all-qualifying names yield MIN-member leaves, and a name whose digest carries $(h+1)k$ leading zero bits — findable by search at small $k$ — raises a boundary at height $h$.

The MIN limit suppresses the single-member spine such names would otherwise chain. At $k \le 2$, where MIN is 1 or 0, it cannot, and the non-reducing-scan stop and height cap bound spines instead; partial single-child nodes inside otherwise-reducing levels still occur. A crafted name distorts shape, not validity. Commit privilege is required, and already suffices to write a flat manifest of arbitrary size — but repositories accepting commits from marginally trusted users should monitor artifact-count growth.