zit 0.2.0

zit 0.2.0 is a capability release. It closes the 0.1.x line, which grew from 38 dispatched verbs to 61, gained the sub-manifest dialect, and grew a layered derived-state cache — and it says so, which the 0.1.x notes largely did not.

Why the minor bump, and why this document is long

0.1.4's notes covered Git interchange, sparse operation, and release portability well. They mention sub-manifests once and the word cache not at all, in a line that by then had already shipped a delta-manifest dialect, a zero-parse image, a rebuild checkpoint, and six other memo tiers.

That is a documentation debt, not a code one: the work was reviewed, tested, and pinned. But a capability nobody can find is close to a capability that does not exist, so 0.2.0 pays the debt down. Much of what follows describes machinery that has been in your repository for several point releases and has never been written up properly. Where that is true, this document says so rather than presenting old work as new.

The version is 0.2.0 rather than 0.1.5 because the accumulated surface — 23 new verbs, an alternate manifest dialect, and a cache hierarchy with its own invariants — is a different product from 0.1.0, and the number should say that.

Highlights

• Sub-manifests: an opt-in delta-manifest dialect that stores a check-in as a delta against a chosen baseline instead of a full file list, with the baseline count k selected by replaying the repository's own history.
• A derived-state ladder: nine memo tiers between the content log and a query, every one of them rebuildable, and every one of them obeying the same rule about what a cache is permitted to answer.
• 61 dispatched verbs, up from 38 at 0.1.0 — interchange, archives, search, moderation, and repository maintenance that 0.1.x shipped incrementally.
• Merge gained two independent conveniences: -X ours|theirs for wholesale conflict resolution, and zit precedent (alias rerere) for replaying a resolution you already authored.
• A real fix: the stash sidecar could be committed and synced. It could not be, before you upgrade; it cannot be, after.

Sub-manifests

A conventional Fossil manifest lists every file in the check-in. For a large tree with small commits, that means each check-in restates a file list that barely changed — the cost of a commit scales with the size of the repository rather than the size of the change.

A sub-manifest stores the check-in as a delta against a baseline manifest, carried in R cards. The dialect is zit's own; it is not something a stock fossil client mints, which is why the whole feature is behind an explicit opt-in.

It is off unless you turn it on. The setting is false on every repository that has not been opted in by an operator act (zit submanifest enable). A repository that never enables it keeps byte-identical check-ins to 0.1.x.

Choosing the baseline interval. The parameter that decides the trade — how many check-ins share a baseline before a new full manifest is minted — is not guessed. zit submanifest analyze replays the repository's own history and reports the k that would have been cheapest for the commits you actually made. It accepts -R over a bare store, so the analysis does not need a working tree.

Commit-time drift advice. With sub-manifests enabled and pinned, a commit that drifts more than 15% from the analyzed optimum prints one advisory line and offers to retune. It never affects whether the commit succeeds — advice, not a gate.

Import. import-fossil chooses deliberately between two readings of the same history: --compatible preserves the original corpus byte-for-byte, so hashes and project code survive a round trip; --submanifest re-mints the history in the dialect, which changes artifact names and is therefore a new repository rather than the same one. import-git mints sub-manifests by default and takes --no-submanifests to produce flat manifests instead.

The cost of reading them. A delta manifest has to be resolved against its baseline before it can be read, so a naive implementation pays that cost per check-in per query. update, verify, and the import path share one sub-manifest memo across every commit in a run instead of rebuilding it, and the memo evicts by recency rather than freezing once it reaches its budget.

The derived-state ladder

The content log is the only authority in a zit repository. Everything else — every index, image, checkpoint, and memo — is derived from it and can be deleted. This section describes what the layers are, because 0.1.x shipped them without naming them.

The rule every layer obeys

A cache that is absent must degrade. A cache that is present must never answer differently from the log.

The failure this rule exists to prevent is the quiet one. A missing cache costs time and nothing else: the query falls back to the log and is correct. A partially correct cache is worse than no cache at all, because it answers confidently and wrongly, and nothing in the output says which layer produced the answer. Every tier below is testable against a full re-derivation, and --no-cache on the derived-state readers bypasses every sidecar and re-derives from the whole log, re-verifying integrity as it goes.

What the tiers are

Memos of derived state

• The zero-parse image — dense, integer-addressed columns holding the resolved check-in graph, so a query reads a slice instead of parsing artifacts. ls, cat, bisect, info, branch, and tag read it directly; log walks its adjacency the way fossil walks its own.
• The crosslink record body — the parsed-record rung beneath the image, used where the image does not carry the answer.
• The rebuild checkpoint — a durable prefix so a rebuild resumes instead of restarting. It accepts delta manifests, anchors at the newest check-in, and builds in one pass with incremental advance.

Memos of log structure

• The log-scan index — the content log's scan index, persisted, so the lazy content channel stops re-walking the log to find where records begin.

Memos of content bytes

• The plaintext memo — reconstructed content held across a run, which frames streams through rather than re-inflating.
• The sub-manifest memo — described above.
• Delta base selection — the memoized bases the log walk would otherwise re-resolve for every manifest.

Memos of the working tree

• The materialization plan, the store cone (with loss-heals-to-dense semantics), and the worktree registry.

What it is worth, measured honestly

On the same repository, one listing costs:

path	time
full derive from the log	29.63s
record-body replay	0.34s
the zero-parse image	0.04s

Having a cache is worth about 87×, and the image a further 8.5×.

An earlier revision of the caching write-up reported 641.9s / 58.3s / 0.31s and claimed 188×. Those were Debug-build numbers compared against a release build — the comparison was invalid, and the figures above replace them, measured in the mode that ships. The embedded page at www/caching.html records the correction rather than quietly restating the numbers, because a performance claim that was once wrong is worth being able to check.

What the image does not carry

The image is not a second source of truth. Queries it cannot answer fall through to the rung below, and the arrangement is asserted rather than assumed: the cache tests compare a .fast_read image load against a full derivation and require query-equivalence, including the adjacency and tag surfaces.

The verb surface

0.1.0 dispatched 38 verbs. 0.2.0 dispatches 61. Added across the line:

• Interchange — import-git, export-git, import-fossil
• Archives — tarball, zip
• Search and inspection — grep (including -S/-G pickaxe over history), frames, verify
• Sparse and multi-tree operation — cone, worktrees
• Store maintenance — checkpoint, submanifest, purge, shun
• Local workflow — stash, precedent
• Configuration — config, settings, trust
• Fossil's other surfaces — wiki, ticket, forum, technote

The command reference (www/reference.html, served on fossil's own /doc/VERSION/PATH route) is pinned by tests in both directions. It had drifted for weeks before those pins existed — grep, purge, config, and trust all shipped undocumented, and nothing noticed, because noticing required a human to compare two lists.

0.2.0 tightens the pin from verbs to flags, holding every verb's real --help against its reference section in both directions: a flag the verb accepts that the section never names, and a flag the section documents that --help never mentions. The second direction matters more than it looks — an omission is discoverable from --help, but a promise sends someone hunting for something that is not there.

That audit immediately found two gaps the section-level pin could not see. pack --drop-elided — the sparse specification's narrow-then-compact deletion, destructive and irreversible — appeared in no help output at all, so nothing had anything to compare it against. It is now documented, including the part that matters most: unlike shun it leaves no tombstone, so a later widening re-fetches the content from a peer. settings --global was accepted but absent from the reference.

Merge

zit merge -X ours|theirs resolves every content conflict in favour of one side. A region only one side changed is still that side's under either strategy, so -X is not "discard their work" — it is "stop asking me". It deliberately does not resolve modify-vs-delete: that is a disagreement about whether a file should exist, not about its contents, and either reading silently destroys work. Git declines it too.

-X ours is not --backout, and the help says so outright: --backout reverses NAME's change; -X ours keeps your side where the two collide and reverses nothing.

zit precedent (alias zit rerere) remembers how you resolved a conflict and replays it when the identical conflict recurs. A merge records each conflict; the commit that resolves it attaches the resolution; a later merge whose conflicted bytes match uses it instead of writing markers.

• The key is the file path and the conflicted bytes together, which is exact: the diff3 envelope carries all three sides, so no segmentation or normalization is involved.
• It is whole-file, so a file whose other conflict moved will not replay. That under-fires deliberately — a resolution spliced into a neighbourhood that changed would be a plausible wrong answer written silently.
• Every replay is reported by path. zit precedent list|show|forget makes the memory inspectable and removable, and zit merge --no-precedent declines replay for one merge without forgetting anything.

The two compose without either knowing about the other: precedent is a pass over the merge outcome set, so a strategy that leaves no conflict leaves it nothing to replay or record. Verified in both directions.

Fixes

The stash sidecar could become an artifact. .zit-stash was absent from the control-path registry, so extras listed it and add accepted it. An ordinary zit add . followed by a commit put the contents of stashed work into a check-in and synced it to every peer — recoverable only by shunning. The stash help had promised "never an artifact" since the feature shipped; nothing enforced it. It is now a reserved authoritative name: invisible to extras, refused by add, and pinned by a test.

zit export-git --help printed a line from another verb's help. A verb-list entry had been pasted into its usage block, backslash and all. The new pin checks shape rather than text — no rendered help line may begin with a backslash — so the whole class is caught for every verb, including verbs added later.

Compatibility

• No artifact-format change. Nothing in this release alters the bytes of a stored artifact, so hashes, history, and existing repositories are unaffected.
• No wire change. The sync protocol is unchanged, and a stock fossil client interoperates exactly as it did with 0.1.4.
• Sub-manifests remain opt-in. A repository that has not enabled them mints byte-identical check-ins to 0.1.x. A repository that has enabled them is using a zit dialect, deliberately.
• New local sidecars. .zit-precedent joins .zit-cache as a reserved advisory name, and .zit-stash is now reserved as authoritative. All are local: never synced, never artifacts, invisible to fossil. Deleting an advisory sidecar costs a rebuild and never an answer.

Qualification

Each published binary is built from the same checksum-pinned source and qualified before publication:

• formatting, and a build at the optimization level that ships;
• the unit suite and the end-to-end suite, which drive the real binary;
• deterministic simulation and the fuzz targets;
• the fossil interop smoke lane, which drives a real fossil binary — the only step that proves the conformance claim against the reference implementation rather than against our own reading of the RFCs, and which now asserts fossil's version rather than merely printing it;
• the packaged binary executed on a deliberately low-feature CPU model for its architecture, because 0.1.3 shipped an x86-64 artifact that used AVX-512 and died with SIGILL on any machine without it.

The release manifest records the artifact checksums, the source identity, the effective target, and the qualification performed for each.