{{- /* The landing page. A page of its own, outside Hextra: one dark identity. The hero figure, the conformance table and every number on the page come from data/*.json and site params, written by scripts/gen-docs.sh and scripts/sync-version.sh; nothing here is typed by hand that the code knows. */ -}} {{- $engine := hugo.Data.engine -}} {{- $conf := hugo.Data.conformance -}} {{- $facts := hugo.Data.facts -}} {{- $words := dict "1" "One" "2" "Two" "3" "Three" "4" "Four" "5" "Five" "6" "Six" "7" "Seven" "8" "Eight" "9" "Nine" -}} {{- $fossil := $facts.fossilVersion -}} {{- $goVersion := replaceRE `^(\d+\.\d+).*$` "$1" site.Params.goVersion -}} {{- $css := resources.Get "css/landing.css" | minify | fingerprint -}} go-libfossil · a Fossil repository engine in Go
go-libfossil v{{ site.Params.version }}

A Fossil repository engine that compiles to WebAssembly.

go-libfossil reads, writes and syncs Fossil repositories: the artifact format, the derived tables and the sync protocol, in Go without cgo. The same engine runs as a command-line tool, inside a server, in a browser tab, or inside a storage object at the edge.

Get startedRead the concepts
$ go get github.com/danmestas/go-libfossil@latest
{{- with $engine }} {{- $m := .manifest }} {{- $changed := slice }} {{- range .rows.mlink }}{{ $changed = $changed | append .file }}{{ end }}
one check-in, three views · output of a real go-libfossil run{{ cond (eq (len $m.name) 64) "sha3-256" "sha1" }} · fossil {{ $fossil }} schema · sync protocol

1 · The artifact

A check-in is a text manifest. Its name is the hash of its bytes, so anything that names it pins its exact content.

{{- $labels := dict "C" "comment" "D" "time" "R" "tree MD5" "U" "user" "Z" "card MD5" "P" "parent check-in" }} {{- range split (strings.TrimSuffix "\n" $m.text) "\n" }} {{- $card := substr . 0 1 }} {{- $fields := split (substr . 2) " " }} {{- $shown := slice }} {{- range $fields }}{{ $shown = $shown | append (partial "landing/short.html" .) }}{{ end }} {{- $label := index $labels $card | default "" }} {{- if eq $card "F" }}{{ $label = cond (in $changed (index $fields 0)) "file → blob" "unchanged" }}{{ end }}
{{ $card }}{{ delimit $shown " " }}{{ with $label }} {{ . }}{{ end }}
{{- end }}
name = {{ cond (eq (len $m.name) 64) "sha3-256" "sha1" }}({{ $m.bytes }} bytes) =
{{ $m.name }}

2 · The repository

Storing it derives the rows fossil derives, in fossil's own tables. fossil rebuild on the same blobs writes the same rows.

blob rid · uuid · size
{{ range .rows.blob }}
{{ .rid }} · {{ partial "landing/short.html" .uuid }} · {{ .size }}
{{ end }}
event type · objid · user · comment
{{ range .rows.event }}
{{ .type }} · {{ .objid }} · {{ .user }} · {{ .comment }}
{{ end }}
plink pid → cid · primary
{{ range .rows.plink }}
{{ .pid }} → {{ .cid }} · {{ .isprim }}
{{ end }}
mlink mid · fid · pid · file
{{ range .rows.mlink }}
{{ .mid }} · {{ .fid }} · {{ .pid }} · {{ .file }}
{{ end }}
tagxref tag · value · rid
{{ range .rows.tagxref }}
{{ .tag }} · {{ .value }} · {{ .rid }}
{{ end }}
leaf rid
{{ range .rows.leaf }}
{{ . }}
{{ end }}

3 · The wire

{{- $rounds := len .pull }}

A clone that has check-in {{ $m.parent.rid }} pulls. Each side announces what it holds; the client asks for what it lacks. {{ index $words (string $rounds) | default $rounds }} round{{ if ne $rounds 1 }}s{{ end }}.

{{- $announced := false }} {{- range $i, $round := .pull }}
→ round {{ add $i 1 }}
{{- range $round.request }} {{- $f := split . " " }} {{- $rest := slice }} {{- range after 1 $f }}{{ $rest = $rest | append (partial "landing/short.html" .) }}{{ end }}
{{ index $f 0 }} {{ delimit $rest " " }}{{ if and (eq (index $f 0) "igot") (eq (index $f 1) $m.parent.uuid) }} # check-in {{ $m.parent.rid }}{{ end }}
{{- end }}
← reply
{{- range $round.reply }} {{- $f := split . " " }} {{- $rest := slice }} {{- range after 1 $f }}{{ $rest = $rest | append (partial "landing/short.html" .) }}{{ end }} {{- $new := and (not $announced) (eq (index $f 0) "igot") (eq (index $f 1) $m.name) }} {{- if $new }}{{ $announced = true }}{{ end }}
{{ index $f 0 }} {{ delimit $rest " " }}{{ if $new }} # new{{ end }}
{{- end }} {{- end }}
native · one static binary, no cgo
js/wasm · SQLite itself in wasm
TinyGo · engine 0.85 MB gzipped
{{- end }}
Where it runs

Version control inside the application, not beside it.

The repository is a SQLite database with Fossil's own tables, so wherever SQLite runs, so does the repository. A database/sql driver is the only thing a new host has to supply.

Server and CLI

One static binary

The libfossil command and the Go API build to a single binary for Linux, macOS and Windows, with no fossil installation and no C toolchain.

Serve repositories, mirror them, or read history from a build pipeline through typed Go values instead of parsed command output.

Browser

A repository in the page

Built for js/wasm with the ncruces SQLite driver, which ships SQLite itself as WebAssembly. A page holds a whole repository and syncs it with a server when it chooses to.

A document that explains a protocol can let its reader commit, branch, merge and sync in the page, and watch the tables change.

Edge storage

History next to the data

Built with TinyGo, the engine fits inside a storage object that runs its own SQLite. Each request is one call into the engine inside one transaction, so it completes or leaves nothing behind.

A publishing tool can keep every revision of what it publishes as Fossil check-ins in the object that serves it, and show the history without a separate service.

Size

Small enough to ship with the page.

The repository engine built with TinyGo and one wasm-opt -Oz pass is under a megabyte compressed. The standard Go toolchain builds the same code about three times larger, which is still useful for debugging.

The TinyGo build covers the repository engine: storage, commits, history, merging, verification. The built-in HTTP sync client needs the standard toolchain today; a TinyGo host supplies its own transport.

TinyGo 0.43, gzipped0.85 MB
Standard Go, gzipped2.4 MB
TinyGo 0.43, raw2.44 MB
Standard Go, raw≈ 9 MB
How we know it is right

Checked against the specification, requirement by requirement.

Fossil's behaviour is written down as a set of RFC drafts. The test suite turns each draft into a table of requirements and runs at least one case against every row; this table is that test's output, {{ $conf.covered }} of {{ $conf.requirements }} requirements met.

{{- $notes := dict "artifact" "every card type, parse and write" "delta" "compared with fossil's own output" "repostate" "derived tables match fossil rebuild" "sync" "client and server" "submanifest" "extension, shared with zit" "sparsecone" "extension, shared with zit" "horizon" "extension, shared with zit" }}
{{- range $conf.suites }} {{- end }}
DraftRequirementsMetNotes
{{ .title }}{{ .requirements }}{{ .covered }}{{ index $notes .suite | safeHTML }}{{ with .partial }}; {{ . }} partial, its scope in the table{{ end }}{{ with .failing }}; {{ . }} failing{{ end }}{{ with .gap }}; {{ . }} without a case{{ end }}

Against the real implementations

Interop tests run the fossil {{ $fossil }} binary and zit against go-libfossil in both directions: each clones, pulls and pushes the other's repositories, and each verifies what it received.

Deterministic simulation

Sync is also run in a seeded simulation that drops requests, truncates replies and partitions nodes. After each run it checks blob integrity, delta chains and convergence. Any failure replays exactly from its seed. The repository also carries {{ lower (index $words (string $facts.fuzzTargets) | default (string $facts.fuzzTargets)) }} fuzz targets.

Partial history

Hold the history you need.

A repository does not have to hold everything. A cone keeps the content of chosen paths. A horizon keeps history from a chosen check-in forward and leaves the rest on the origin, fetched only when an operation needs it. At the structural depth the clone also drops the manifests below the horizon and derives branch and tag state from a seed the origin signs.

A structural clone of fossil-scm.org's repository, 21,008 check-ins, takes about 30 seconds. How horizons work.

horizon Con the originheld C trunk feature release
heldleft on the originhorizon, with a signed seed
Start

Open a repository and commit.

From the command line, or from Go. The libfossil command drives the same code as the library, and the file it writes is an ordinary Fossil repository: stock fossil reads it.

Get startedAPI reference
shell · the check-in name depends on the time
$ go install github.com/danmestas/go-libfossil/cmd/libfossil@latest
$ libfossil repo new notes.fossil
created repository: notes.fossil
$ mkdir work && cd work
$ libfossil -R ../notes.fossil repo open .
$ echo '# notes' > README.md
$ libfossil repo add README.md
ADDED    README.md
$ libfossil repo ci -m "first notes" --user ana
checkin <name> (rid=2)
$ fossil timeline -R ../notes.fossil
<time> [<name>] first notes (user: ana tags: trunk)
main.go
import (
  "github.com/danmestas/go-libfossil"
  _ "github.com/danmestas/go-libfossil/db/driver/modernc"
)

r, err := libfossil.Create("notes.fossil", libfossil.CreateOpts{User: "ana"})
// handle err
rid, uuid, err := r.Commit(libfossil.CommitOpts{
  Files:   []libfossil.FileToCommit{{"{{"}}Name: "README", Content: readme}},
  Comment: "first notes",
  User:    "ana",
})
// fossil open notes.fossil sees the same check-in.
What it is not
  • Not a replacement for the fossil binary's web interface, wiki or forum server.
  • Not a hosting service: it is a library and a command-line tool you run.
  • Not a Git bridge. It speaks Fossil's formats and protocol only.