# zit 0.7.0
zit 0.7.0 adds horizon clones: a clone that holds a repository's history
from a chosen check-in forward, and fetches what lies below it only
when asked. It ships the draft that specifies them, and it changes how a
cone-scoped origin serves a delta that crosses the cone and how a cone
clone is written to disk.
This release is from trunk. A store at a horizon cannot be opened by
older releases; see Compatibility.
## Horizon clones
A horizon is a check-in C and a rule for which check-ins count as at or
after it. The held set is computed over P cards, never over log order
or timestamps. Under the default rule, `descendants`, it is C and every
check-in descended from C. Under `leaves` it also takes in, for every
childless check-in that does not descend from C, everything descended
from where its ancestry meets C's. That keeps the merge base of two live
branches inside the clone, and on a repository with old open branches it
costs a good deal more. Every artifact a horizon clone holds is the
artifact the origin holds, under the same name. Nothing is rewritten,
and what the clone lacks it knows by name as elided phantoms.
There are two kinds.
The content horizon keeps every check-in manifest, every control
artifact, wiki, ticket, technote, forum and attachment artifact of every
age. It keeps file content only for the check-ins in the held set.
Timeline, leaves, branches, tags and merge ancestry are derived exactly
as on a dense clone, because every input to the derive is present.
The structural horizon also leaves out the check-in manifests and tree
nodes below C, keeping only those the held set reaches through P, B and
S cards. A delta manifest's baseline below C is kept, because the
manifest cannot be expanded without it. Control artifacts are still kept
at every age. In place of the missing ancestry the origin supplies a
seed (below), so branch and tag state above C still derives exactly. The
timeline ends at C.
```
zit clone URL [DIR] --horizon=HASH [--horizon-rule=descendants|leaves]
zit clone URL [DIR] --horizon=HASH --structural
```
`--horizon` takes the full hash of C, and `--structural` is meaningless
without it. A clone reports what the origin did:
```
cloned 2986 artifacts into usr.bin in 5 rounds (horizon clone; origin served: structural; nothing elided locally)
```
An operation that needs history below C says so and names the horizon,
rather than treating the missing artifact as damage or as an empty file.
`cat`, `diff` and `annotate` name the elision. `zit update CHECKIN`
with a hash below C deepens: it moves C back, keeps the depth and rule,
and pulls what the new held set adds from the saved remote. `merge`
asks the origin for a merge base below C, deepens to it, and runs over
real artifacts. A deepen with no known common ancestor of C and the
target is refused, and C does not move. `zit verify` counts deliberately
elided artifacts on their own line, distinct from faults. `zit
export-git` refuses a horizon store, because a Git mirror needs every blob.
A new plumbing verb, `zit merge-base A B`, prints Fossil's merge base of
two check-ins. An origin answers a horizon clone's merge-base question
with the same computation.
Measured on NetBSD src (317,081 check-ins, 3,649 MB dense), served over
loopback, every clone verifying with 0 faults: a structural clone of the
whole tree at the trunk tip holds 182,735 artifacts and takes 189 s. On
the pkgsrc sub-manifest store a structural clone is 61 MB against
632 MB dense.
## Horizons and path cones
A horizon is a member of the store cone, so `--horizon` combines with
`--cone`, `--cone-not` and `--cone-file`. A blob is kept when it is both
at or after the horizon and named by an in-cone F card. The horizon
alone decides which check-in manifests are held. Within the trees it
holds, the origin withholds the sub-manifest nodes that no S-card slot
reaching into the cone leads to. Under a content horizon that walk
covers every check-in's tree, so the origin walks only the trees
appended since it last classified the cone.
On NetBSD src at the trunk tip, with every clone verifying clean:
| Clone | Wall |
|---|---|
| structural, `--cone=usr.bin`, cold | 4.6 s |
| structural, `--cone=usr.bin`, warm | 1.10 s |
| structural, `--cone=sys` | 44.5 s |
| content, `--cone=usr.bin` | 452 s, 449,113 artifacts, 359,052,984-byte store |
| content, `--cone=sys` | 1611 s, 713,630 artifacts |
## The seed
The seed is a control artifact the origin mints from its own derivation.
Its T cards apply directly to C the tags in force there: the branch tag
with its value, every other propagating tag, the closed marker when C is
closed, and every non-propagating tag on C. It does the same for each
other check-in of the held set whose primary parent lies below the
horizon, which is what a branch forked below C and later merged with C
needs. The D card is the latest time among the tag events it summarizes,
rounded up to a whole second, so a seed over unchanged state has the
same bytes and the same name every time it is minted. The U card is
`zit-serve`, not a user's login.
The clone holds the seed beside the store cone, outside the artifact
store, and applies it at every open as an ordinary tag application. It
never sends it to any peer. It names the seed it holds on every pull or
sync, and the origin sends a new one only when its name differs. A
repository with no tag rows in force has nothing to seed: the origin
mints none and serves the request at the content level.
The seed feeds an artifact in one place. A check-in that creates a
branch writes a cancel card for the parent's branch as the clone
derived it. An origin checks that card at push. If the clone's seed was
stale, it refuses the check-in and names the command that corrects it,
and it admits the check-in once the same push carries that correction.
## Against a stock Fossil server
Fossil ignores the horizon pragmas and serves densely. A horizon clone
of a Fossil server classifies what arrives locally, keeps what the
content horizon keeps and records the rest as elided, so it costs
bandwidth and never correctness. A structural request has no seed to
stand on, so the clone falls back to the content horizon and says so:
```
zit clone: the origin did not serve the structural horizon with a seed; keeping the content horizon instead
```
Fossil folds unclustered artifacts into a cluster and announces only
the cluster, so a horizon clone pulling from Fossil requests what a
cluster names and classifies it on arrival. Against a zit origin that
says `horizon-served`, it requests nothing that only a cluster names.
## Pulling and syncing from a horizon clone
`pull` and `sync` carry the store's horizon on every request. Pass no
flag: the origin filters by the same held set, labels boundary parents,
re-seeds when needed, and sends what new check-ins add. The origin
hands back a cursor, so a later pull announces nothing and a push
announces only what the clone appended. On the pkgsrc structural clone,
a no-op pull is 431 bytes up and 237 down. The origin keeps no record
of which clients hold what.
A structural store that pulls or syncs from a peer whose replies do not
carry `horizon-served structural`, such as stock Fossil, falls back to
the content horizon at the same C and rule. It drops its seed, fetches
the manifests below C that the content horizon holds, and says so:
```
zit pull: the origin did not serve the structural horizon; keeping the content horizon instead, with every manifest below it
```
A horizon clone is not a complete origin. Served, it refuses a dense
clone and any horizon clone it cannot cover, with one error card and no
content. A stock Fossil clone of it ends in a refusal and leaves no
repository behind. A dense peer that already holds the history below
the horizon can still pull from it.
## Pushing to stock Fossil
A check-in made in a horizon clone is an ordinary check-in. Its manifest
names its parent and its files by real hash, and nothing in it reveals
the horizon. Check-ins made in a content horizon clone of a Fossil
server and in a structural horizon clone of a zit server, pushed to a
stock Fossil server, arrive byte for byte as zit wrote them. A fresh
Fossil clone of that server rebuilds and integrity-checks clean and
checks each one out to the files zit committed.
## Streaming persist for cone and horizon clones
A cone or horizon clone used to hold every round in memory until the
exchange ended, then filter the whole transfer locally and write it. It
now does that only when it must. When the origin's first reply says it
filtered for this request, the rounds are written to the log as they
arrive, the way a dense clone's are, and each round's bytes are released
once written. For a horizon the origin says `horizon-served` at the
depth asked for, and for a request naming paths it says `cone-served`.
The cone is then classified once over the log. An origin that did not
filter still gets the batched local filter.
## Crossing deltas
An origin may store a kept blob as a delta against a base that the
requester's cone elides. Served verbatim, it is a blob the recipient
holds and can never expand. A cone-scoped or horizon-scoped origin now
checks each served delta against the request's keep set and sends one
whose base is outside it in full form, as sparse-cone section 6.2 and
horizon section 6.2.1 require. Only deltas that cross are rebuilt. The
recipient still treats the source of a retained delta as a transit
phantom, so an origin that cannot rebuild one costs bandwidth, never a
stuck artifact.
On a path-only `--cone=usr.bin` clone of NetBSD src, 0.6.6's origin sent
1563 out-of-cone delta bases that the client fetched and then dropped.
0.7.0 sends none.
## Path-cone classification
The classifier's last step re-walked every tree in the store to find
sub-manifest nodes it could elide by slot. A clone from an origin that
sends every node has no such node, so the walk found nothing. It now runs
only when some S card names a node the store does not hold.
The same path-only `usr.bin` clone of NetBSD src, 932,336 artifacts:
| Build | Wall | Client user CPU | Store |
|---|---|---|---|
| zit 0.6.6 | 1826 s | 1347 s | 1,244,509,552 bytes |
| zit 0.7.0 | 1713 s | 1343 s | 1,241,666,920 bytes |
In both, about 55% of the wall time is the cache derive after the
transfer.
## Cloning a cone from an older origin
An origin from 0.6.3 through 0.6.6 acknowledges a path cone with
`cone-served`, so a 0.7.0 client streams its clone. That origin predates
the crossing-delta rule: it sends a kept blob as a delta against an
out-of-cone base, and answers the clone's request for the base.
Streamed, that base would have been written to the store as it landed.
What arrives after the clone sequence is now held back for the cone to
judge. A base the cone does not keep is dropped, and the kept blob is
stored in full form, as the batched path already did.
## Wire capabilities
An origin that implements the horizon draft says `pragma horizon-served
content` or `pragma horizon-served structural` in every reply to a
request carrying a horizon, naming the deepest cut it served. It
advertises `structural` only when the same reply carries the seed, or
the client already holds the seed it would send. A client enters the
structural horizon only on that word, and a structural store leaves it
on any exchange that lacks it. `cone-served`, from 0.6.3, now also
decides whether a path-cone clone streams. The other pragmas are
`horizon`, `horizon-structural`, `horizon-below`, `horizon-seed`,
`horizon-mergebase` and `horizon-cursor`. A merge-base query carries at
most two secondaries, the checkout and one pending merge, and a merge
that needs more across a horizon is refused.
Every one is a pragma, which Fossil and zit 0.6.x ignore. Against such a
peer a horizon request degrades as described above. A path-cone request
to an origin without `cone-served` is filtered locally, as before.
## The horizon draft
The source package ships `docs/rfcs/draft-fossil-horizon-00.txt`. It
specifies the horizon as a temporal member of the sparse cone, both cuts,
the seed, boundary labels, re-seeding, commit and push across a stale
seed, deepen, the sync pragmas, the origin's classification kept across
requests, the cursor, and degradation against a peer that knows none of
it. The sparse-cone draft states the crossing-delta rule and the member
kinds a cone may carry. The sync-protocol draft lists every horizon
pragma with its direction.
## Compatibility
Dense and path-cone stores are unchanged and stay readable by every
0.6.x release. A store at a horizon carries a new store header version,
written before the horizon policy and lowered only once no horizon is
held. 0.6.x refuses such a store, and its derived image, rather than
read it as dense. Deepening, re-seeding and `pack` keep the horizon
version. The origin's persisted classification and structural graph are
`.zit-cache` sidecars: deleting them costs time, never data.
On the wire, a 0.7.0 origin is an ordinary origin to stock Fossil and to
zit 0.6.x. The horizon pragmas change only a reply to a request that
carries one. A stock Fossil clone of a 0.7.0 server, and later pulls
from it, produce the repository a Fossil server of the same content
does. A 0.7.0 horizon clone works against Fossil or zit 0.6.x at the
content level. It falls back from structural, and pushes check-ins Fossil
stores byte for byte. A path-cone clone from 0.6.x's origin keeps
nothing outside the cone. The Fossil interoperability smoke scenarios
cover each direction.
## Qualification
The release manifest records native qualification against the exact tagged
source archive on macOS ARM64, Linux x86-64 and Linux ARM64, using the pinned
Zig 0.17.0 toolchain. Publication is gated on unit and end-to-end tests, Git
differentials, deterministic simulation, seeded fuzzing, and Fossil
interoperability checks on every distributed target.
For the basis of the conformance claim, see
[the 0.1.0 release notes](RELEASE-NOTES-0.1.0.md).