Artifact bdac26335c2d8d60e50ac3e4a2b390c43e7de706e05075010a2e91617a907103:
- File
README.md
— part of check-in
[b53032d01f]
at
2026-04-24 16:02:07
on branch trunk
— notify: extract shared natshdr.Carrier to kill duplicate header-carrier
Two copies of natsHeaderCarrier existed (leaf/agent/nats.go and leaf/agent/notify/pubsub.go), both carrying the same latent case-sensitive Get bug that was fixed in Task 6. Extract the type to leaf/agent/internal/natshdr so the next bug can be fixed once.
While consolidating, drop the 4-tier fallback in Get in favor of a single EqualFold scan. Header maps are tiny (typically 3 entries) and tiers 1-3 were subsumed by tier 4 anyway. The new implementation is demonstrably correct and easier to read. (user: [email protected], size: 5941) [annotate] [blame] [check-ins using] [more...]
# EdgeSync Replace Fossil's HTTP sync with NATS messaging or direct peer-to-peer sync. Leaf agents can act as both sync clients and servers — a stock `fossil clone`/`fossil sync` can talk to them directly. ## Architecture ```mermaid graph LR Leaf[Leaf Agent] subgraph Bridged Bridge[Bridge] --> Fossil[Fossil Server] end subgraph Peer-to-Peer Peer[Other Leaf] end Leaf -- NATS --> Bridge Leaf -- NATS --> Peer Leaf -- HTTP --> Peer Leaf -- "NATS over iroh (QUIC)" --> Peer ``` **Four sync modes:** 1. **Leaf → Bridge → Fossil Server** — Original mode. Bridge translates NATS to HTTP `/xfer`. 2. **Leaf → Leaf (NATS)** — Peer-to-peer via ServeNATS. No bridge or server needed. 3. **Leaf → Leaf (HTTP)** — Peer-to-peer via ServeHTTP. Stock `fossil clone`/`fossil sync` works. 4. **Leaf → Leaf (iroh)** — Peer-to-peer over QUIC with NAT traversal. Each agent runs an embedded NATS server; leaf node connections tunnel over iroh. Enables presence and messaging without central infrastructure. ## Quick Start ```bash # One command: install hooks + build + test make setup # Run the leaf agent bin/leaf --repo my.fossil --nats nats://localhost:4222 --serve-http :8080 # Run the edgesync CLI bin/edgesync repo info ``` ### Dual VCS Note This repo is tracked by both Git and Fossil. Go's VCS stamping gets confused by this, so **all `go build` commands need `-buildvcs=false`**. The Makefile handles this automatically. If you run `go build` directly: ```bash # Either pass the flag every time: go build -buildvcs=false ./cmd/edgesync/ # Or set it once per shell session: export GOFLAGS="-buildvcs=false" go build ./cmd/edgesync/ # works without the flag now ``` If you use [direnv](https://direnv.net/), the included `.envrc` sets this for you automatically. ### Telemetry (Optional) Traces, metrics, and structured logs via OpenTelemetry. Zero overhead when disabled — the leaf agent logs whether telemetry is active at startup. ```bash # With Doppler (manages OTel secrets): doppler run -- bin/leaf --repo my.fossil --nats nats://localhost:4222 # Or set the endpoint directly: OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4318 bin/leaf --repo my.fossil --nats nats://localhost:4222 ``` ## Project Layout ``` cmd/edgesync/ Unified CLI binary (50 subcommands) libfossil/ Core library — Go port of Fossil internals annotate/ Line-level blame/annotate bisect/ Binary search for regressions blob/ Blob compression (Fossil's 4-byte prefix + zlib) content/ Artifact storage and expansion (delta chains) db/ SQLite adapter (3 drivers via build tags) deck/ Manifest/control-artifact parsing delta/ Fossil delta codec (ported from delta.c) hash/ SHA1/SHA3-256 content addressing manifest/ Checkin, file listing, timeline merge/ 3-way merge with swappable strategies path/ Checkout path resolution repo/ Fossil repo DB operations (create, open, verify) simio/ Simulation I/O abstractions (Clock, Rand, Env) stash/ Working-tree stash save/restore sync/ Sync engine: client (Sync, Clone), server (HandleSync, ServeHTTP), transport (HTTP, NATS, mock) tag/ Tag read/write on artifacts testutil/ Shared test helpers undo/ Undo/redo state tracking xfer/ Xfer card protocol encoder/decoder leaf/ Leaf agent module agent/ Agent logic, NATS transport, ServeNATS, ServeP2P stub cmd/leaf/ Standalone leaf daemon binary bridge/ Bridge module bridge/ Bridge logic (NATS <-> HTTP /xfer translation) cmd/bridge/ Standalone bridge daemon binary dst/ Deterministic simulation testing SimNetwork (bridge mode), PeerNetwork (leaf-to-leaf) sim/ Integration simulation testing Real NATS + Fossil + fault proxy + serve tests cmd/soak/ Continuous soak test runner docs/ Documentation dev/specs/ Design specifications dev/plans/ Implementation plans ``` ## Go Modules | Module | Path | Purpose | |--------|------|---------| | `github.com/danmestas/EdgeSync` | `.` | Root: CLI, sim/, soak runner | | `github.com/danmestas/libfossil` | `libfossil/` | Core library (public, v0.3.x) | | `github.com/danmestas/EdgeSync/leaf` | `leaf/` | Leaf agent | | `github.com/danmestas/EdgeSync/bridge` | `bridge/` | Bridge | | `github.com/danmestas/EdgeSync/dst` | `dst/` | Deterministic sim tests | ## SQLite Drivers ```bash go build ./... # modernc (default, pure Go) go build -tags ncruces ./... # ncruces (WASM-based) CGO_ENABLED=1 go build -tags mattn ./... # mattn (CGo, best performance) ``` ## Testing ```bash make test # CI tests + sim serve (~15s) make dst # DST: 8 seeds, normal (~2s) make dst-full # DST: 16 seeds x 3 levels (~40s) make sim # Integration sim: 1 seed (requires fossil) make sim-full # Integration sim: 16 seeds x 3 severities make drivers # Test all 3 SQLite drivers ``` ## Dependencies - `modernc.org/sqlite` — Pure Go SQLite (default driver) - `github.com/nats-io/nats.go` — NATS + JetStream client - `github.com/alecthomas/kong` — CLI framework - `github.com/hexops/gotextdiff` — Unified diff output ## Reference - [Fossil sync protocol](https://fossil-scm.org/home/doc/tip/www/sync.wiki) - [Fossil delta format](https://fossil-scm.org/home/doc/tip/www/delta_format.wiki) - [Fossil password authentication](https://fossil-scm.org/home/doc/tip/www/password.wiki)