zit 0.7.0 adds horizon clones: a clone that holds a repository's history from a chosen check-in forward, and fetches what lies below it only when asked. It ships the draft that specifies them, and it changes how a cone-scoped origin serves a delta that crosses the cone and how a cone clone is written to disk.
This release is from trunk. A store at a horizon cannot be opened by older releases; see Compatibility.
Horizon clones
A horizon is a check-in C and a rule for which check-ins count as at or
after it. The held set is computed over P cards, never over log order
or timestamps. Under the default rule, descendants, it is C and every
check-in descended from C. Under leaves it also takes in, for every
childless check-in that does not descend from C, everything descended
from where its ancestry meets C's. That keeps the merge base of two live
branches inside the clone, and on a repository with old open branches it
costs a good deal more. Every artifact a horizon clone holds is the
artifact the origin holds, under the same name. Nothing is rewritten,
and what the clone lacks it knows by name as elided phantoms.
There are two kinds.
The content horizon keeps every check-in manifest, every control artifact, wiki, ticket, technote, forum and attachment artifact of every age. It keeps file content only for the check-ins in the held set. Timeline, leaves, branches, tags and merge ancestry are derived exactly as on a dense clone, because every input to the derive is present.
The structural horizon also leaves out the check-in manifests and tree nodes below C, keeping only those the held set reaches through P, B and S cards. A delta manifest's baseline below C is kept, because the manifest cannot be expanded without it. Control artifacts are still kept at every age. In place of the missing ancestry the origin supplies a seed (below), so branch and tag state above C still derives exactly. The timeline ends at C.
zit clone URL [DIR] --horizon=HASH [--horizon-rule=descendants|leaves]
zit clone URL [DIR] --horizon=HASH --structural
--horizon takes the full hash of C, and --structural is meaningless
without it. A clone reports what the origin did:
cloned 2986 artifacts into usr.bin in 5 rounds (horizon clone; origin served: structural; nothing elided locally)
An operation that needs history below C says so and names the horizon,
rather than treating the missing artifact as damage or as an empty file.
cat, diff and annotate name the elision. zit update CHECKIN
with a hash below C deepens: it moves C back, keeps the depth and rule,
and pulls what the new held set adds from the saved remote. merge
asks the origin for a merge base below C, deepens to it, and runs over
real artifacts. A deepen with no known common ancestor of C and the
target is refused, and C does not move. zit verify counts deliberately
elided artifacts on their own line, distinct from faults. zit
export-git refuses a horizon store, because a Git mirror needs every blob.
A new plumbing verb, zit merge-base A B, prints Fossil's merge base of
two check-ins. An origin answers a horizon clone's merge-base question
with the same computation.
Measured on NetBSD src (317,081 check-ins, 3,649 MB dense), served over loopback, every clone verifying with 0 faults: a structural clone of the whole tree at the trunk tip holds 182,735 artifacts and takes 189 s. On the pkgsrc sub-manifest store a structural clone is 61 MB against 632 MB dense.
Horizons and path cones
A horizon is a member of the store cone, so --horizon combines with
--cone, --cone-not and --cone-file. A blob is kept when it is both
at or after the horizon and named by an in-cone F card. The horizon
alone decides which check-in manifests are held. Within the trees it
holds, the origin withholds the sub-manifest nodes that no S-card slot
reaching into the cone leads to. Under a content horizon that walk
covers every check-in's tree, so the origin walks only the trees
appended since it last classified the cone.
On NetBSD src at the trunk tip, with every clone verifying clean:
| Clone | Wall |
|---|---|
structural, --cone=usr.bin, cold |
4.6 s |
structural, --cone=usr.bin, warm |
1.10 s |
structural, --cone=sys |
44.5 s |
content, --cone=usr.bin |
452 s, 449,113 artifacts, 359,052,984-byte store |
content, --cone=sys |
1611 s, 713,630 artifacts |
The seed
The seed is a control artifact the origin mints from its own derivation.
Its T cards apply directly to C the tags in force there: the branch tag
with its value, every other propagating tag, the closed marker when C is
closed, and every non-propagating tag on C. It does the same for each
other check-in of the held set whose primary parent lies below the
horizon, which is what a branch forked below C and later merged with C
needs. The D card is the latest time among the tag events it summarizes,
rounded up to a whole second, so a seed over unchanged state has the
same bytes and the same name every time it is minted. The U card is
zit-serve, not a user's login.
The clone holds the seed beside the store cone, outside the artifact store, and applies it at every open as an ordinary tag application. It never sends it to any peer. It names the seed it holds on every pull or sync, and the origin sends a new one only when its name differs. A repository with no tag rows in force has nothing to seed: the origin mints none and serves the request at the content level.
The seed feeds an artifact in one place. A check-in that creates a branch writes a cancel card for the parent's branch as the clone derived it. An origin checks that card at push. If the clone's seed was stale, it refuses the check-in and names the command that corrects it, and it admits the check-in once the same push carries that correction.
Against a stock Fossil server
Fossil ignores the horizon pragmas and serves densely. A horizon clone of a Fossil server classifies what arrives locally, keeps what the content horizon keeps and records the rest as elided, so it costs bandwidth and never correctness. A structural request has no seed to stand on, so the clone falls back to the content horizon and says so:
zit clone: the origin did not serve the structural horizon with a seed; keeping the content horizon instead
Fossil folds unclustered artifacts into a cluster and announces only
the cluster, so a horizon clone pulling from Fossil requests what a
cluster names and classifies it on arrival. Against a zit origin that
says horizon-served, it requests nothing that only a cluster names.
Pulling and syncing from a horizon clone
pull and sync carry the store's horizon on every request. Pass no
flag: the origin filters by the same held set, labels boundary parents,
re-seeds when needed, and sends what new check-ins add. The origin
hands back a cursor, so a later pull announces nothing and a push
announces only what the clone appended. On the pkgsrc structural clone,
a no-op pull is 431 bytes up and 237 down. The origin keeps no record
of which clients hold what.
A structural store that pulls or syncs from a peer whose replies do not
carry horizon-served structural, such as stock Fossil, falls back to
the content horizon at the same C and rule. It drops its seed, fetches
the manifests below C that the content horizon holds, and says so:
zit pull: the origin did not serve the structural horizon; keeping the content horizon instead, with every manifest below it
A horizon clone is not a complete origin. Served, it refuses a dense clone and any horizon clone it cannot cover, with one error card and no content. A stock Fossil clone of it ends in a refusal and leaves no repository behind. A dense peer that already holds the history below the horizon can still pull from it.
Pushing to stock Fossil
A check-in made in a horizon clone is an ordinary check-in. Its manifest names its parent and its files by real hash, and nothing in it reveals the horizon. Check-ins made in a content horizon clone of a Fossil server and in a structural horizon clone of a zit server, pushed to a stock Fossil server, arrive byte for byte as zit wrote them. A fresh Fossil clone of that server rebuilds and integrity-checks clean and checks each one out to the files zit committed.
Streaming persist for cone and horizon clones
A cone or horizon clone used to hold every round in memory until the
exchange ended, then filter the whole transfer locally and write it. It
now does that only when it must. When the origin's first reply says it
filtered for this request, the rounds are written to the log as they
arrive, the way a dense clone's are, and each round's bytes are released
once written. For a horizon the origin says horizon-served at the
depth asked for, and for a request naming paths it says cone-served.
The cone is then classified once over the log. An origin that did not
filter still gets the batched local filter.
Crossing deltas
An origin may store a kept blob as a delta against a base that the requester's cone elides. Served verbatim, it is a blob the recipient holds and can never expand. A cone-scoped or horizon-scoped origin now checks each served delta against the request's keep set and sends one whose base is outside it in full form, as sparse-cone section 6.2 and horizon section 6.2.1 require. Only deltas that cross are rebuilt. The recipient still treats the source of a retained delta as a transit phantom, so an origin that cannot rebuild one costs bandwidth, never a stuck artifact.
On a path-only --cone=usr.bin clone of NetBSD src, 0.6.6's origin sent
1563 out-of-cone delta bases that the client fetched and then dropped.
0.7.0 sends none.
Path-cone classification
The classifier's last step re-walked every tree in the store to find sub-manifest nodes it could elide by slot. A clone from an origin that sends every node has no such node, so the walk found nothing. It now runs only when some S card names a node the store does not hold.
The same path-only usr.bin clone of NetBSD src, 932,336 artifacts:
| Build | Wall | Client user CPU | Store |
|---|---|---|---|
| zit 0.6.6 | 1826 s | 1347 s | 1,244,509,552 bytes |
| zit 0.7.0 | 1713 s | 1343 s | 1,241,666,920 bytes |
In both, about 55% of the wall time is the cache derive after the transfer.
Cloning a cone from an older origin
An origin from 0.6.3 through 0.6.6 acknowledges a path cone with
cone-served, so a 0.7.0 client streams its clone. That origin predates
the crossing-delta rule: it sends a kept blob as a delta against an
out-of-cone base, and answers the clone's request for the base.
Streamed, that base would have been written to the store as it landed.
What arrives after the clone sequence is now held back for the cone to
judge. A base the cone does not keep is dropped, and the kept blob is
stored in full form, as the batched path already did.
Wire capabilities
An origin that implements the horizon draft says pragma horizon-served
content or pragma horizon-served structural in every reply to a
request carrying a horizon, naming the deepest cut it served. It
advertises structural only when the same reply carries the seed, or
the client already holds the seed it would send. A client enters the
structural horizon only on that word, and a structural store leaves it
on any exchange that lacks it. cone-served, from 0.6.3, now also
decides whether a path-cone clone streams. The other pragmas are
horizon, horizon-structural, horizon-below, horizon-seed,
horizon-mergebase and horizon-cursor. A merge-base query carries at
most two secondaries, the checkout and one pending merge, and a merge
that needs more across a horizon is refused.
Every one is a pragma, which Fossil and zit 0.6.x ignore. Against such a
peer a horizon request degrades as described above. A path-cone request
to an origin without cone-served is filtered locally, as before.
The horizon draft
The source package ships docs/rfcs/draft-fossil-horizon-00.txt. It
specifies the horizon as a temporal member of the sparse cone, both cuts,
the seed, boundary labels, re-seeding, commit and push across a stale
seed, deepen, the sync pragmas, the origin's classification kept across
requests, the cursor, and degradation against a peer that knows none of
it. The sparse-cone draft states the crossing-delta rule and the member
kinds a cone may carry. The sync-protocol draft lists every horizon
pragma with its direction.
Compatibility
Dense and path-cone stores are unchanged and stay readable by every
0.6.x release. A store at a horizon carries a new store header version,
written before the horizon policy and lowered only once no horizon is
held. 0.6.x refuses such a store, and its derived image, rather than
read it as dense. Deepening, re-seeding and pack keep the horizon
version. The origin's persisted classification and structural graph are
.zit-cache sidecars: deleting them costs time, never data.
On the wire, a 0.7.0 origin is an ordinary origin to stock Fossil and to zit 0.6.x. The horizon pragmas change only a reply to a request that carries one. A stock Fossil clone of a 0.7.0 server, and later pulls from it, produce the repository a Fossil server of the same content does. A 0.7.0 horizon clone works against Fossil or zit 0.6.x at the content level. It falls back from structural, and pushes check-ins Fossil stores byte for byte. A path-cone clone from 0.6.x's origin keeps nothing outside the cone. The Fossil interoperability smoke scenarios cover each direction.
Qualification
The release manifest records native qualification against the exact tagged source archive on macOS ARM64, Linux x86-64 and Linux ARM64, using the pinned Zig 0.17.0 toolchain. Publication is gated on unit and end-to-end tests, Git differentials, deterministic simulation, seeded fuzzing, and Fossil interoperability checks on every distributed target.
For the basis of the conformance claim, see the 0.1.0 release notes.