Skip to content
go-libfossil Artifact [73d1faeb15]
Login

Artifact [73d1faeb15]

Check-ins Using Download Text

Artifact 73d1faeb154245965b1f67a5f86fe631d333395f479ea96f53eac177e816afaa:

title: Horizons

weight: 70

A horizon cuts a store by time. It names a check-in C, a rule, and a depth. The check-ins the rule keeps are the held set. A blob is elided when no check-in in the held set names it. Structure (manifests, tags, control artifacts) is never elided by a content horizon.

A horizon is a member of a store cone, so everything on Sparse cones about store cones, elided artifacts and compaction applies to it too. The behaviour follows draft-fossil-horizon-00, implemented in step with zit.

The held set is computed from the check-in graph the repository holds. It is stored nowhere (§3.1).

Rules

A check-in is childless when no other check-in names it as a parent, open or closed. A branch tip merged into another branch has a child, so it is not childless. Under the leaves rule, any two childless check-ins have their merge base in the held set, so merging branches that are still being extended never crosses the horizon. Both rules give a set that includes every child of every check-in in it.

The leaves rule has a cost you should check before using it. An old branch that was never merged forks from C's ancestry near the root, and the rule then holds every descendant of that fork. On fossil-scm.org's repository it held 20,907 of 21,008 check-ins at every depth tried. The descendants rule, with C 10, 100, 426 and 1000 check-ins back from the tip, saves 97, 86, 56 and 41 percent of that history (§3.4). Use the leaves rule when the repository keeps few old unmerged branches.

repo cone cost <check-in> (Repo.HorizonCost) reports what each rule would hold at a check-in, read from a repository that has the history:

$ libfossil -R repo.fossil repo cone cost trunk
horizon at trunk
descendants  1 check-ins, 3 blobs, 6 B expanded, 57 B stored
leaves  1 check-ins, 3 blobs, 6 B expanded, 57 B stored

Depth

A content horizon keeps all structure and elides only blobs; a structural horizon also leaves out the manifests below C.

Seeds are never pushed

A seed is private to the clone that holds it, so a go-libfossil origin refuses any pushed control artifact that looks like one (§5.2.3):

Each test runs on the artifact's content after any delta is applied, including a delta whose base arrives later in the same push. The refusal is atomic: nothing the push carried is stored, and the reply is one error card, seed refused: a seed is private to the clone that holds it; it is never pushed, which a go-libfossil pusher reports among its sync errors. A delta whose base has not arrived cannot be read yet, so it is stored as any other dangling delta is, and judged when a later push supplies the base. Until it is judged it is withheld: never announced or sent. Whatever fills the base, a later push, a pull, a check-in or tag that writes the same bytes, judges it in the same transaction, and the crosslink step judges anything still pending before it links it. The push or pull that supplied the base is stored as usual, with no error, and the seed is recorded as refused in a table of the store's own that no peer pulls: it is never crosslinked (a rebuild included), announced or sent. A seed-shaped artifact that was never pushed, one pulled from a peer, say, is ordinary.

Two consequences follow:

Signed seeds

An origin with the clearsign setting on (and the legacy omitsign off) signs each seed it mints with its pgp-command, the same settings fossil and zit sign check-ins with (§5.2.4). It keeps one signature per seed, so a seed keeps its name while the tag state it records is unchanged.

A clone given the origin's public key verifies every seed against it: at clone time, on every pull, and when it rebuilds. A seed that is unsigned, or signed by another key, is discarded and the clone falls back to the content horizon. A clone with no key accepts an unsigned seed. The key is the clone's own horizon-seed-key setting, never something the origin sends. CloneOpts.SeedKey and repo clone --seed-key set it when the clone is made; Repo.SetConfig refuses a value that is neither an SSH public key nor an OpenPGP fingerprint.

An origin signs only for a requester that may read, after the request is authorized, and keeps the signature in its own table; a store that cannot write, such as a read-only mirror, signs again for every request. Setting the origin's own horizon-seed-key to its public key makes it check a kept signature before serving it, so replacing the key behind an unchanged pgp-command takes effect at once. A clone that cannot run its verifier (gpg missing, or timed out) keeps the seed it holds and reports the error rather than falling back. Settings are read from the repository only: go-libfossil reads no global fossil configuration, so clearsign and pgp-command set with fossil settings --global do not apply.

Signing covers the signing settings and the verifier in more detail.

Moving the horizon

Moving C earlier and pulling fetches what the deeper horizon keeps. Moving it later keeps what you hold until you compact. A horizon store cannot be a checkout's cone: a horizon governs what the repository holds, not which paths a checkout writes.

Library usage

Build a horizon cone

h := libfossil.Horizon{
    Checkin:    "9f2c...",                   // full 40 or 64 digit hash
    Rule:       libfossil.HorizonDescendants, // or HorizonLeaves
    Structural: false,                        // true for the structural cut
}

// Horizon only: every path, cut by time.
hc, err := libfossil.NewHorizonCone(nil, nil, h)

// Horizon plus paths.
hc, err = libfossil.NewHorizonCone([]string{"src"}, nil, h)

got, ok := hc.Horizon()               // read it back
noHorizon, err := hc.WithHorizon(nil) // drop it, keep the paths

Checkin must be a full hash, and it is lower-cased. Dropping the horizon of a horizon-only cone is an error, since nothing would be left; use SetStoreCone(nil) to make the store dense.

Clone into a horizon store

t := libfossil.NewHTTPTransport("https://example.com/repo")
repo, res, err := libfossil.Clone(ctx, "repo.fossil", t, libfossil.CloneOpts{
    StoreCone: hc,
    SeedKey:   "ssh-ed25519 AAAA...", // optional; structural only
})
if err != nil {
    log.Fatal(err)
}
defer repo.Close()
fmt.Println(res.Elided, res.HorizonServed, res.StructuralServed, res.HorizonFallback)

SyncResult has the same fields.

HorizonHistoryURL, on both results, is the address the origin gives for querying history below the horizon outside the sync protocol, when it gives one. An origin sets it with repo config set horizon-history-url https://... (SettingHorizonHistoryURL); only one http or https URL with no spaces or credentials is accepted. repo clone prints it as History:.

Deepen, merge across, and verify the seed

Setting, replacing and dropping a horizon on an existing store use SetStoreCone and SetStoreConeContext, as for any store cone.

// Move the horizon down far enough to reach a check-in, which may lie
// below it. C moves to that check-in when it is an ancestor of C, else to
// a common ancestor of the two, asked of the origin when the store cannot
// see one. The rule, the depth and the path members stay.
sr, err := repo.DeepenHorizon(ctx, target, libfossil.RequiredContentOpts{Peer: t})

// Merge. With a Peer, a merge where either side's ancestry crosses a
// structural horizon asks the origin for the base, deepening to it when it
// lies below. Without one it fails with an error matching
// libfossil.ErrBelowHorizon, even when the store finds a base it holds.
mr, err := co.MergeContext(ctx, libfossil.CheckoutMergeOpts{Version: rid},
    libfossil.RequiredContentOpts{Peer: t})

// Check the seed against the history it summarizes. This fetches every
// manifest below C and leaves a content-horizon store.
v, err := repo.VerifySeed(ctx, libfossil.RequiredContentOpts{Peer: t})

Deepening a structural store discards its seed before it fetches the new one, so the old seed cannot shadow the new one. If the transfer fails, the old horizon and seed are put back (§5.8).

A merge never runs against a base the store does not hold. A bisection whose good end lies below the horizon is refused the same way, naming the horizon and the deepen.

An origin checks each pushed check-in that creates a branch: if the branch it cancels is not the one this origin derives for the parent, the clone's seed was stale, and the whole push is refused with stale seed: ... (§5.7). Pull, push a control artifact that cancels the parent's real branch tag on the check-in, and push again.

Command line

Clone a horizon store

libfossil repo clone URL repo.fossil --horizon <full hash>
libfossil repo clone URL repo.fossil --horizon <full hash> --leaves --cone src
libfossil repo clone URL repo.fossil --horizon <full hash> --structural --seed-key "ssh-ed25519 AAAA..."

--cone and --cone-not combine with these. The clone needs a full hash because nothing is local yet to resolve a name against.

Change the horizon

libfossil -R repo.fossil repo cone store --horizon <check-in>
libfossil -R repo.fossil repo cone store --horizon <check-in> --leaves
libfossil -R repo.fossil repo cone store --horizon <check-in> --structural
libfossil -R repo.fossil repo cone store --no-horizon
libfossil -R repo.fossil repo cone deepen <check-in>
libfossil -R repo.fossil repo cone cost <check-in>

Merge and verify

libfossil -R repo.fossil repo merge <version> -d DIR
libfossil -R repo.fossil repo verify --seed

store, deepen, merge and verify take --remote (default: the saved last-sync-url), --sync-user, --sync-pass (never saved), --sync-private and --timeout (default 10m; 0 disables it).

Interoperability

zit. The horizon format matches zit's byte for byte. Horizon sync works in both directions against zit's horizon-rfc revision.

Stock fossil 2.28. A horizon store refuses a dense clone, because the clone would be silently incomplete.

A peer that knows nothing of horizons. A horizon clone from stock Fossil works. The client asks for nothing below the horizon, records the below-horizon blobs it is sent as elided instead of storing them, and classifies locally. HorizonServed stays 0. Stock Fossil can announce a new check-in only inside a cluster. A horizon store holds no clusters, so it requests the cluster members as transit and classifies what arrives (§4.5). Without that a new check-in would not be learned until a later one named it.

Limits