Skip to content
zit Artifact [232e102c02]
Login

Artifact [232e102c02]

Check-ins Using Download Hex Wiki

Artifact 232e102c02a7e5010684833ea4dcc11e95a18bf147cf5097bf5153ccb04bbea4:

# zit 0.2.0

zit 0.2.0 is a capability release. It closes the 0.1.x line, which grew from
38 dispatched verbs to 61, gained the sub-manifest dialect, and grew a
layered derived-state cache — and it says so, which the 0.1.x notes largely
did not.

## Why the minor bump, and why this document is long

0.1.4's notes covered Git interchange, sparse operation, and release
portability well. They mention sub-manifests once and the word *cache* not at
all, in a line that by then had already shipped a delta-manifest dialect, a
zero-parse image, a rebuild checkpoint, and six other memo tiers.

That is a documentation debt, not a code one: the work was reviewed, tested,
and pinned. But a capability nobody can find is close to a capability that
does not exist, so 0.2.0 pays the debt down. Much of what follows describes
machinery that has been in your repository for several point releases and has
never been written up properly. Where that is true, this document says so
rather than presenting old work as new.

The version is 0.2.0 rather than 0.1.5 because the accumulated surface — 23
new verbs, an alternate manifest dialect, and a cache hierarchy with its own
invariants — is a different product from 0.1.0, and the number should say
that.

## Highlights

- **Sub-manifests**: an opt-in delta-manifest dialect that stores a check-in
  as a delta against a chosen baseline instead of a full file list, with the
  baseline count `k` selected by replaying the repository's own history.
- **A derived-state ladder**: nine memo tiers between the content log and a
  query, every one of them rebuildable, and every one of them obeying the
  same rule about what a cache is permitted to answer.
- **61 dispatched verbs**, up from 38 at 0.1.0 — interchange, archives,
  search, moderation, and repository maintenance that 0.1.x shipped
  incrementally.
- **Merge gained two independent conveniences**: `-X ours|theirs` for
  wholesale conflict resolution, and `zit precedent` (alias `rerere`) for
  replaying a resolution you already authored.
- **A real fix**: the stash sidecar could be committed and synced. It could
  not be, before you upgrade; it cannot be, after.

## Sub-manifests

A conventional Fossil manifest lists every file in the check-in. For a large
tree with small commits, that means each check-in restates a file list that
barely changed — the cost of a commit scales with the size of the repository
rather than the size of the change.

A sub-manifest stores the check-in as a delta against a baseline manifest,
carried in R cards. The dialect is zit's own; it is not something a stock
fossil client mints, which is why the whole feature is behind an explicit
opt-in.

**It is off unless you turn it on.** The setting is `false` on every
repository that has not been opted in by an operator act
(`zit submanifest enable`). A repository that never enables it keeps
byte-identical check-ins to 0.1.x.

**Choosing the baseline interval.** The parameter that decides the trade —
how many check-ins share a baseline before a new full manifest is minted —
is not guessed. `zit submanifest analyze` replays the repository's own
history and reports the `k` that would have been cheapest for the commits you
actually made. It accepts `-R` over a bare store, so the analysis does not
need a working tree.

**Commit-time drift advice.** With sub-manifests enabled and pinned, a commit
that drifts more than 15% from the analyzed optimum prints one advisory line
and offers to retune. It never affects whether the commit succeeds — advice,
not a gate.

**Import.** `import-fossil` chooses deliberately between two readings of the
same history: `--compatible` preserves the original corpus byte-for-byte, so
hashes and project code survive a round trip; `--submanifest` re-mints the
history in the dialect, which changes artifact names and is therefore a new
repository rather than the same one. `import-git` mints sub-manifests by
default and takes `--no-submanifests` to produce flat manifests instead.

**The cost of reading them.** A delta manifest has to be resolved against its
baseline before it can be read, so a naive implementation pays that cost per
check-in per query. `update`, `verify`, and the import path share one
sub-manifest memo across every commit in a run instead of rebuilding it, and
the memo evicts by recency rather than freezing once it reaches its budget.

## The derived-state ladder

The content log is the only authority in a zit repository. Everything else —
every index, image, checkpoint, and memo — is derived from it and can be
deleted. This section describes what the layers are, because 0.1.x shipped
them without naming them.

### The rule every layer obeys

**A cache that is absent must degrade. A cache that is present must never
answer differently from the log.**

The failure this rule exists to prevent is the quiet one. A missing cache
costs time and nothing else: the query falls back to the log and is correct.
A *partially* correct cache is worse than no cache at all, because it answers
confidently and wrongly, and nothing in the output says which layer produced
the answer. Every tier below is testable against a full re-derivation, and
`--no-cache` on the derived-state readers bypasses every sidecar and
re-derives from the whole log, re-verifying integrity as it goes.

### What the tiers are

**Memos of derived state**

- *The zero-parse image* — dense, integer-addressed columns holding the
  resolved check-in graph, so a query reads a slice instead of parsing
  artifacts. `ls`, `cat`, `bisect`, `info`, `branch`, and `tag` read it
  directly; `log` walks its adjacency the way fossil walks its own.
- *The crosslink record body* — the parsed-record rung beneath the image,
  used where the image does not carry the answer.
- *The rebuild checkpoint* — a durable prefix so a rebuild resumes instead of
  restarting. It accepts delta manifests, anchors at the newest check-in, and
  builds in one pass with incremental advance.

**Memos of log structure**

- *The log-scan index* — the content log's scan index, persisted, so the lazy
  content channel stops re-walking the log to find where records begin.

**Memos of content bytes**

- *The plaintext memo* — reconstructed content held across a run, which
  `frames` streams through rather than re-inflating.
- *The sub-manifest memo* — described above.
- *Delta base selection* — the memoized bases the log walk would otherwise
  re-resolve for every manifest.

**Memos of the working tree**

- *The materialization plan*, *the store cone* (with loss-heals-to-dense
  semantics), and *the worktree registry*.

### What it is worth, measured honestly

On the same repository, one listing costs:

| path | time |
|---|---|
| full derive from the log | 29.63s |
| record-body replay | 0.34s |
| the zero-parse image | 0.04s |

Having a cache is worth about 87×, and the image a further 8.5×.

An earlier revision of the caching write-up reported 641.9s / 58.3s / 0.31s
and claimed 188×. Those were Debug-build numbers compared against a release
build — the comparison was invalid, and the figures above replace them,
measured in the mode that ships. The embedded page at `www/caching.html`
records the correction rather than quietly restating the numbers, because a
performance claim that was once wrong is worth being able to check.

### What the image does not carry

The image is not a second source of truth. Queries it cannot answer fall
through to the rung below, and the arrangement is asserted rather than
assumed: the cache tests compare a `.fast_read` image load against a full
derivation and require query-equivalence, including the adjacency and tag
surfaces.

## The verb surface

0.1.0 dispatched 38 verbs. 0.2.0 dispatches 61. Added across the line:

- **Interchange** — `import-git`, `export-git`, `import-fossil`
- **Archives** — `tarball`, `zip`
- **Search and inspection** — `grep` (including `-S`/`-G` pickaxe over
  history), `frames`, `verify`
- **Sparse and multi-tree operation** — `cone`, `worktrees`
- **Store maintenance** — `checkpoint`, `submanifest`, `purge`, `shun`
- **Local workflow** — `stash`, `precedent`
- **Configuration** — `config`, `settings`, `trust`
- **Fossil's other surfaces** — `wiki`, `ticket`, `forum`, `technote`

The command reference (`www/reference.html`, served on fossil's own
`/doc/VERSION/PATH` route) is pinned by tests in both directions. It had
drifted for weeks before those pins existed — `grep`, `purge`, `config`, and
`trust` all shipped undocumented, and nothing noticed, because noticing
required a human to compare two lists.

0.2.0 tightens the pin from verbs to flags, holding every verb's real
`--help` against its reference section in both directions: a flag the verb
accepts that the section never names, and a flag the section documents that
`--help` never mentions. The second direction matters more than it looks — an
omission is discoverable from `--help`, but a promise sends someone hunting
for something that is not there.

That audit immediately found two gaps the section-level pin could not see.
`pack --drop-elided` — the sparse specification's narrow-then-compact
deletion, destructive and irreversible — appeared in no help output at all,
so nothing had anything to compare it against. It is now documented,
including the part that matters most: unlike `shun` it leaves no tombstone,
so a later widening re-fetches the content from a peer. `settings --global`
was accepted but absent from the reference.

## Merge

**`zit merge -X ours|theirs`** resolves every *content* conflict in favour of
one side. A region only one side changed is still that side's under either
strategy, so `-X` is not "discard their work" — it is "stop asking me". It
deliberately does not resolve modify-vs-delete: that is a disagreement about
whether a file should exist, not about its contents, and either reading
silently destroys work. Git declines it too.

`-X ours` is not `--backout`, and the help says so outright: `--backout`
reverses NAME's change; `-X ours` keeps your side where the two collide and
reverses nothing.

**`zit precedent`** (alias `zit rerere`) remembers how you resolved a
conflict and replays it when the identical conflict recurs. A merge records
each conflict; the commit that resolves it attaches the resolution; a later
merge whose conflicted bytes match uses it instead of writing markers.

- The key is the file path and the conflicted bytes together, which is exact:
  the diff3 envelope carries all three sides, so no segmentation or
  normalization is involved.
- It is whole-file, so a file whose *other* conflict moved will not replay.
  That under-fires deliberately — a resolution spliced into a neighbourhood
  that changed would be a plausible wrong answer written silently.
- Every replay is reported by path. `zit precedent list|show|forget` makes the
  memory inspectable and removable, and `zit merge --no-precedent` declines
  replay for one merge without forgetting anything.

The two compose without either knowing about the other: precedent is a pass
over the merge outcome set, so a strategy that leaves no conflict leaves it
nothing to replay or record. Verified in both directions.

## Fixes

**The stash sidecar could become an artifact.** `.zit-stash` was absent from
the control-path registry, so `extras` listed it and `add` accepted it. An
ordinary `zit add .` followed by a commit put the *contents of stashed work*
into a check-in and synced it to every peer — recoverable only by shunning.
The stash help had promised "never an artifact" since the feature shipped;
nothing enforced it. It is now a reserved authoritative name: invisible to
`extras`, refused by `add`, and pinned by a test.

**`zit export-git --help` printed a line from another verb's help.** A
verb-list entry had been pasted into its usage block, backslash and all. The
new pin checks shape rather than text — no rendered help line may begin with
a backslash — so the whole class is caught for every verb, including verbs
added later.

## Compatibility

- **No artifact-format change.** Nothing in this release alters the bytes of
  a stored artifact, so hashes, history, and existing repositories are
  unaffected.
- **No wire change.** The sync protocol is unchanged, and a stock fossil
  client interoperates exactly as it did with 0.1.4.
- **Sub-manifests remain opt-in.** A repository that has not enabled them
  mints byte-identical check-ins to 0.1.x. A repository that has enabled them
  is using a zit dialect, deliberately.
- **New local sidecars.** `.zit-precedent` joins `.zit-cache` as a reserved
  advisory name, and `.zit-stash` is now reserved as authoritative. All are
  local: never synced, never artifacts, invisible to fossil. Deleting an
  advisory sidecar costs a rebuild and never an answer.

## Qualification

Each published binary is built from the same checksum-pinned source and
qualified before publication:

- formatting, and a build at the optimization level that ships;
- the unit suite and the end-to-end suite, which drive the real binary;
- deterministic simulation and the fuzz targets;
- the fossil interop smoke lane, which drives a real fossil binary — the only
  step that proves the conformance claim against the reference implementation
  rather than against our own reading of the RFCs, and which now asserts
  fossil's version rather than merely printing it;
- the packaged binary executed on a deliberately low-feature CPU model for
  its architecture, because 0.1.3 shipped an x86-64 artifact that used
  AVX-512 and died with SIGILL on any machine without it.

The release manifest records the artifact checksums, the source identity, the
effective target, and the qualification performed for each.