Skip to content
infra Update of ”NATS Server — Setup +amp; Architecture”
Login

Update of ”NATS Server — Setup +amp; Architecture”

History Page Raw
Overview

Artifact ID: 8fccb6390ec3589feea20810a732b2488b218e701288b0409e8b305fa7f173f2
Page Name:NATS Server — Setup & Architecture
Date: 2026-03-15 20:27:27
Original User: dmestas
Mimetype:text/x-markdown
Parent: ca9cc569520239ac7f73ae44224891038f2095cb0ac5d987dbc78e0d6814ddd1 (diff)
Content

NATS Server — Setup & Architecture

NATS 2.10 JetStream server for the Fossil NATS Sync project. Provides durable pub/sub messaging between Fossil repositories and their leaf-node agents.

Public endpoint: tls://sync.craftdesign.group:4222

Architecture

Leaf Node TLS :4222 nats-server Account JWT JetStream Admin CLI acme.sh cert Cloudflare DNS DNS-01 EDGESYNC GO_LIBFOSSIL 
    right

    # External clients
    C1: box "Leaf Node" color white fill 0x3a5a3a width 1.2 height 0.4 rad 4px
    move 0.1
    arrow "TLS :4222" above color 0x555555 thick
    move 0.1
    NS: box "nats-server" color white fill 0x2a4a8a width 1.4 height 0.4 rad 4px
    move 0.1
    arrow color 0x555555
    move 0.1
    AC: box "Account JWT" fill 0xddeeff width 1.4 height 0.4 rad 4px
    move 0.1
    arrow color 0x555555
    move 0.1
    JS: box "JetStream" color white fill 0x2a6b8a width 1.3 height 0.4 rad 4px

    # Second client below
    C2: box "Admin CLI" color white fill 0x3a5a3a width 1.2 height 0.4 rad 4px \
        at C1.s + (0, -0.7)
    arrow from C2.e right 0.1 then to NS.sw color 0x555555

    # TLS cert path below nats-server
    down
    TLS: box "acme.sh" fill 0xd0f0d0 width 1.0 height 0.35 rad 4px at NS.s + (0, -0.7)
    arrow from TLS.n to NS.s color 0x888888 <- "cert" rjust small
    CF: box "Cloudflare DNS" fill 0xd0f0d0 width 1.4 height 0.35 rad 4px at TLS.e + (0.6, 0)
    arrow <-> from TLS.e to CF.w color 0x888888 "DNS-01" above small

    # Streams below JetStream
    ES: box "EDGESYNC" fill 0xddeeff width 1.3 height 0.35 rad 4px at JS.s + (0, -0.6)
    GL: box "GO_LIBFOSSIL" fill 0xddeeff width 1.3 height 0.35 rad 4px at JS.s + (0, -1.1)
    arrow from JS.s to ES.n color 0x888888
    arrow from JS.s down 0.05 then to GL.n color 0x888888

What Is Installed

Component Version Path
nats-server 2.10.22 /usr/local/bin/nats-server
nats CLI 0.1.4 /usr/local/bin/nats
nsc 2.9.0 ~/.local/bin/nsc (dmestas)
acme.sh latest /root/.acme.sh/acme.sh
TLS cert Let's Encrypt ECC /var/nats/tls/sync.craftdesign.group.{crt,key}

Key Paths

Path Purpose
/usr/local/bin/nats-server Server binary
/etc/nats/nats-server.conf Live rendered config
/etc/nats/nats-server.conf.template Config template (envsubst vars)
/etc/nats/operator.env Operator JWT (root-owned, 0600)
/etc/nats/cf-token.env Cloudflare API token (root-owned, 0600)
/var/nats/jetstream/ JetStream message storage
/var/nats/jwt/ Account JWT resolver directory
/var/nats/tls/ TLS cert + key
~/Projects/infra/nats/creds/ .creds files for each user
~/.local/share/nats/nsc/ nsc operator/account/key store (dmestas)
/etc/systemd/system/nats.service Systemd unit

Auth Hierarchy

Operator: fossil-sync SYS edgesync go-libfossil admin bridge admin bridge pub/sub: > fossil.edgesync.> pub/sub: > fossil.go-libfossil.> 
    down

    OP: box "Operator: fossil-sync" fill 0xfff0cc width 2.4 height 0.45 rad 5px
    move 0.3

    # Three accounts side by side
    SYS: box "SYS" fill 0xd0eeff width 0.9 height 0.4 rad 4px at OP.s + (-1.5, -0.5)
    ES:  box "edgesync" fill 0xd0f0d0 width 1.1 height 0.4 rad 4px at OP.s + (0, -0.5)
    GL:  box "go-libfossil" fill 0xd0f0d0 width 1.3 height 0.4 rad 4px at OP.s + (1.6, -0.5)

    line from OP.s to SYS.n color 0x888888
    line from OP.s to ES.n  color 0x888888
    line from OP.s to GL.n  color 0x888888

    # Users under edgesync
    ESA: box "admin" fill 0xeeffee width 0.9 height 0.35 rad 4px at ES.s + (-0.55, -0.55)
    ESB: box "bridge" fill 0xeeffee width 0.9 height 0.35 rad 4px at ES.s + (0.55, -0.55)
    line from ES.s to ESA.n color 0x888888
    line from ES.s to ESB.n color 0x888888

    # Users under go-libfossil
    GLA: box "admin" fill 0xeeffee width 0.9 height 0.35 rad 4px at GL.s + (-0.55, -0.55)
    GLB: box "bridge" fill 0xeeffee width 0.9 height 0.35 rad 4px at GL.s + (0.55, -0.55)
    line from GL.s to GLA.n color 0x888888
    line from GL.s to GLB.n color 0x888888

    # Annotations
    text "pub/sub: >" small at ESA.s + (0, -0.2)
    text "fossil.edgesync.>" small at ESB.s + (0, -0.2)
    text "pub/sub: >" small at GLA.s + (0, -0.2)
    text "fossil.go-libfossil.>" small at GLB.s + (0, -0.2)

JetStream Streams

Stream Subject filter Retention Max size
EDGESYNC fossil.edgesync.> 7 days 1 GB
GO_LIBFOSSIL fossil.go-libfossil.> 7 days 1 GB

Storage: file (/var/nats/jetstream/) Replicas: 1 (single-node) Discard policy: old (drop oldest when full) Dupe window: 2 minutes

Install Scripts (4 Phases)

All scripts live in nats/ in the infra Fossil repo.

Phase 1: install.sh  (root) binary · nats user · dirs · TLS cert · systemd unit Phase 2: bootstrap-creds.sh  (dmestas) nsc operator · accounts · users · .creds files · operator.env Phase 3: start.sh  (root) render config · start service · copy account JWTs Phase 4: provision-streams.sh  (dmestas) create EDGESYNC and GO_LIBFOSSIL streams 
    down

    P1: box "Phase 1: install.sh  (root)" fill 0xfff0cc width 3.2 height 0.45 rad 5px
    text "binary · nats user · dirs · TLS cert · systemd unit" small at P1.s + (0, -0.18)
    arrow from P1.s down 0.28 color 0x888888
    P2: box "Phase 2: bootstrap-creds.sh  (dmestas)" fill 0xd0f0d0 width 3.2 height 0.45 rad 5px
    text "nsc operator · accounts · users · .creds files · operator.env" small at P2.s + (0, -0.18)
    arrow from P2.s down 0.28 color 0x888888
    P3: box "Phase 3: start.sh  (root)" fill 0xd0eeff width 3.2 height 0.45 rad 5px
    text "render config · start service · copy account JWTs" small at P3.s + (0, -0.18)
    arrow from P3.s down 0.28 color 0x888888
    P4: box "Phase 4: provision-streams.sh  (dmestas)" fill 0xd0f0ff width 3.2 height 0.45 rad 5px
    text "create EDGESYNC and GO_LIBFOSSIL streams" small at P4.s + (0, -0.18)

Running a fresh install

# On the server, as dmestas:
cd ~/Projects/infra

# Phase 1 (root) — idempotent
export NATS_VERSION=2.10.22
sudo --preserve-env=NATS_VERSION bash nats/install.sh

# Phase 2 (dmestas) — idempotent
bash nats/bootstrap-creds.sh

# Phase 3 (root) — idempotent
sudo bash nats/start.sh

# Phase 4 (dmestas)
bash nats/provision-streams.sh

Prerequisites for Phase 1:

Prerequisites for Phase 2:

Firewall

UFW is used (not Hetzner Cloud firewall).

sudo ufw allow 4222/tcp   # NATS TLS (already open)
sudo ufw status

Port 4222 is the only NATS-related port that needs to be open externally.