Skip to content
infra Changes To NATS Server — Setup & Architecture
Login

Changes To NATS Server — Setup & Architecture

Previous

Changes to "NATS Server — Setup & Architecture" between 2026-03-15 20:21:57 and 2026-03-15 20:27:27

1
2
3
4
5
6
7
8
9
10
11
12

13

14
15
16


17
18
19

20
21

22
23
24
25
26



27
28
29
30
31
32





33
34


35
36
37
38
39
40
41
42
43
44
45
46
47
48




49
50
51
52





53
54

55
56

57
58




59
60
61
62
63
64
65

1
2
3
4
5
6
7
8
9
10
11

12

13
14


15
16



17


18





19
20
21






22
23
24
25
26


27
28



29










30
31
32
33
34



35
36
37
38
39


40
41

42


43
44
45
46
47
48
49
50
51
52
53












-
+
-
+

-
-
+
+
-
-
-
+
-
-
+
-
-
-
-
-
+
+
+
-
-
-
-
-
-
+
+
+
+
+
-
-
+
+
-
-
-

-
-
-
-
-
-
-
-
-
-
+
+
+
+

-
-
-
+
+
+
+
+
-
-
+

-
+
-
-
+
+
+
+








# NATS Server — Setup & Architecture

NATS 2.10 JetStream server for the **Fossil NATS Sync** project. Provides durable
pub/sub messaging between Fossil repositories and their leaf-node agents.

Public endpoint: `tls://sync.craftdesign.group:4222`

---

## Architecture

```pikchr
``` pikchr
scale = 0.85
    right

# External clients (left column)
box "External Leaf Node" "(e.g. dev laptop)" fit rad 6px at (0, 2.4)
    # External clients
    C1: box "Leaf Node" color white fill 0x3a5a3a width 1.2 height 0.4 rad 4px
box "External Leaf Node" "(e.g. CI runner)"  fit rad 6px at (0, 1.2)
box "Local Admin"        "(nats CLI)"         fit rad 6px at (0, 0.0)

    move 0.1
# Arrow labels
arrow from 1st box.e right 0.3 then right 0.9; "TLS :4222" above
    arrow "TLS :4222" above color 0x555555 thick
arrow from 2nd box.e right 0.3 then right 0.9; "TLS :4222" above
arrow from 3rd box.e right 0.3 then right 0.9; "TLS :4222" above

# NATS server box
NX = 2.8
    move 0.1
    NS: box "nats-server" color white fill 0x2a4a8a width 1.4 height 0.4 rad 4px
    move 0.1
NY = 1.2
box wid 2.2 ht 2.8 rad 6px at (NX, NY) "nats-server 2.10" "(vps-nbg1)" big bold fill lightyellow

# JetStream
box "JetStream" fit rad 4px at (NX, 1.8) fill lightblue
arrow from (NX, 1.55) down 0.25
    arrow color 0x555555
    move 0.1
    AC: box "Account JWT" fill 0xddeeff width 1.4 height 0.4 rad 4px
    move 0.1
    arrow color 0x555555
box "/var/nats/jetstream" fit rad 4px at (NX, 1.2) fill lightyellow

    move 0.1
    JS: box "JetStream" color white fill 0x2a6b8a width 1.3 height 0.4 rad 4px
# JWT resolver
arrow from (NX, 0.95) down 0.25
box "/var/nats/jwt" fit rad 4px at (NX, 0.6) fill lightyellow

# Accounts (right column)
RX = 5.4
box "Account: edgesync"     fit rad 6px at (RX, 2.2) fill lightgreen
box "  users: admin, bridge" fit rad 4px at (RX, 1.8) fill honeydew
box "Account: go-libfossil"  fit rad 6px at (RX, 1.1) fill lightgreen
box "  users: admin, bridge" fit rad 4px at (RX, 0.7) fill honeydew
box "Account: SYS"           fit rad 6px at (RX, 0.1) fill lightyellow

arrow from (NX+1.1, 2.0) right 0.3; "" above
arrow from (NX+1.1, 1.0) right 0.3; "" above
    # Second client below
    C2: box "Admin CLI" color white fill 0x3a5a3a width 1.2 height 0.4 rad 4px \
        at C1.s + (0, -0.7)
    arrow from C2.e right 0.1 then to NS.sw color 0x555555

# TLS / cert renewal (bottom)
box "acme.sh" fit rad 6px at (1.4, -0.8) fill lightcyan
box "Cloudflare DNS" fit rad 6px at (3.4, -0.8) fill lightcyan
    # TLS cert path below nats-server
    down
    TLS: box "acme.sh" fill 0xd0f0d0 width 1.0 height 0.35 rad 4px at NS.s + (0, -0.7)
    arrow from TLS.n to NS.s color 0x888888 <- "cert" rjust small
    CF: box "Cloudflare DNS" fill 0xd0f0d0 width 1.4 height 0.35 rad 4px at TLS.e + (0.6, 0)
arrow from (1.4, -0.55) up 0.2 then to (NX-0.4, -0.1) "cert" above
arrow <-> from (1.4+0.7, -0.8) right 0.6; "DNS-01" above
    arrow <-> from TLS.e to CF.w color 0x888888 "DNS-01" above small

# systemd (top)
    # Streams below JetStream
box "systemd nats.service" fit rad 6px at (NX, 3.2) fill lavender
arrow from (NX, 3.0) down 0.15
    ES: box "EDGESYNC" fill 0xddeeff width 1.3 height 0.35 rad 4px at JS.s + (0, -0.6)
    GL: box "GO_LIBFOSSIL" fill 0xddeeff width 1.3 height 0.35 rad 4px at JS.s + (0, -1.1)
    arrow from JS.s to ES.n color 0x888888
    arrow from JS.s down 0.05 then to GL.n color 0x888888
```

---

## What Is Installed

| Component | Version | Path |
︙
86
87
88
89
90
91
92
93

94

95
96
97


98
99
100
101
102




103
104
105
106



107
108
109
110
111
112





113
114
115
116
117
118





119
120
121
122
123
124





125
126
127
128
129
130
131

74
75
76
77
78
79
80

81

82
83


84
85
86




87
88
89
90
91



92
93
94
95





96
97
98
99
100
101





102
103
104
105
106
107





108
109
110
111
112
113
114
115
116
117
118
119








-
+
-
+

-
-
+
+

-
-
-
-
+
+
+
+

-
-
-
+
+
+

-
-
-
-
-
+
+
+
+
+

-
-
-
-
-
+
+
+
+
+

-
-
-
-
-
+
+
+
+
+








| `~/.local/share/nats/nsc/` | nsc operator/account/key store (dmestas) |
| `/etc/systemd/system/nats.service` | Systemd unit |

---

## Auth Hierarchy

```pikchr
``` pikchr
scale = 0.8
    down

# Operator
box "Operator: fossil-sync" wid 3.2 ht 0.5 rad 6px at (2.5, 3.0) fill lightyellow bold
    OP: box "Operator: fossil-sync" fill 0xfff0cc width 2.4 height 0.45 rad 5px
    move 0.3

# Accounts row
box "SYS" wid 1.2 ht 0.5 rad 6px at (0.8, 2.0) fill lightcyan
box "edgesync" wid 1.4 ht 0.5 rad 6px at (2.5, 2.0) fill lightgreen
box "go-libfossil" wid 1.6 ht 0.5 rad 6px at (4.4, 2.0) fill lightgreen
    # Three accounts side by side
    SYS: box "SYS" fill 0xd0eeff width 0.9 height 0.4 rad 4px at OP.s + (-1.5, -0.5)
    ES:  box "edgesync" fill 0xd0f0d0 width 1.1 height 0.4 rad 4px at OP.s + (0, -0.5)
    GL:  box "go-libfossil" fill 0xd0f0d0 width 1.3 height 0.4 rad 4px at OP.s + (1.6, -0.5)

line from (2.5, 2.75) to (0.8, 2.25)
line from (2.5, 2.75) to (2.5, 2.25)
line from (2.5, 2.75) to (4.4, 2.25)
    line from OP.s to SYS.n color 0x888888
    line from OP.s to ES.n  color 0x888888
    line from OP.s to GL.n  color 0x888888

# Users under edgesync
box "admin" wid 1.0 ht 0.4 rad 4px at (1.9, 1.0) fill honeydew
box "bridge" wid 1.0 ht 0.4 rad 4px at (3.1, 1.0) fill honeydew
line from (2.5, 1.75) to (1.9, 1.2)
line from (2.5, 1.75) to (3.1, 1.2)
    # Users under edgesync
    ESA: box "admin" fill 0xeeffee width 0.9 height 0.35 rad 4px at ES.s + (-0.55, -0.55)
    ESB: box "bridge" fill 0xeeffee width 0.9 height 0.35 rad 4px at ES.s + (0.55, -0.55)
    line from ES.s to ESA.n color 0x888888
    line from ES.s to ESB.n color 0x888888

# Users under go-libfossil
box "admin" wid 1.0 ht 0.4 rad 4px at (3.8, 1.0) fill honeydew
box "bridge" wid 1.0 ht 0.4 rad 4px at (5.0, 1.0) fill honeydew
line from (4.4, 1.75) to (3.8, 1.2)
line from (4.4, 1.75) to (5.0, 1.2)
    # Users under go-libfossil
    GLA: box "admin" fill 0xeeffee width 0.9 height 0.35 rad 4px at GL.s + (-0.55, -0.55)
    GLB: box "bridge" fill 0xeeffee width 0.9 height 0.35 rad 4px at GL.s + (0.55, -0.55)
    line from GL.s to GLA.n color 0x888888
    line from GL.s to GLB.n color 0x888888

# Labels
text "pub/sub: >" at (2.3, 0.55) small
text "pub/sub: fossil.edgesync.>" at (2.5, 0.55) small
text "pub/sub: >" at (4.2, 0.55) small
text "pub/sub: fossil.go-libfossil.>" at (4.4, 0.55) small
    # Annotations
    text "pub/sub: >" small at ESA.s + (0, -0.2)
    text "fossil.edgesync.>" small at ESB.s + (0, -0.2)
    text "pub/sub: >" small at GLA.s + (0, -0.2)
    text "fossil.go-libfossil.>" small at GLB.s + (0, -0.2)
```

- **admin** users: full `>` pub/sub (local provisioning only)
- **bridge** users: scoped to `fossil.<account>.>` (bridge daemon)
- **SYS** account: NATS system events (server-internal use)

---
︙
144
145
146
147
148
149
150
151

152
153
154
155
156

157
158
159
160
161
162







163
164
165
166
167


168
169
170

171
172
173
174
175
176
177
178
179

180
181
182
183
184
185
186
187

132
133
134
135
136
137
138

139





140
141





142
143
144
145
146
147
148





149
150



151









152

153
154
155
156
157
158
159








-
+
-
-
-
-
-
+

-
-
-
-
-
+
+
+
+
+
+
+
-
-
-
-
-
+
+
-
-
-
+
-
-
-
-
-
-
-
-
-
+
-









---

## Install Scripts (4 Phases)

All scripts live in `nats/` in the `infra` Fossil repo.

```pikchr
``` pikchr
scale = 0.85
X = 0.0
W = 3.0
H = 0.7
G = 0.3
    down

box "Phase 1: install.sh" wid W ht H rad 6px at (X, 3*H+3*G) fill lightyellow
  text "run as root" small at last box.s + (0,0.1)
box "Phase 2: bootstrap-creds.sh" wid W ht H rad 6px at (X, 2*H+2*G) fill lightgreen
  text "run as dmestas" small at last box.s + (0,0.1)
box "Phase 3: start.sh" wid W ht H rad 6px at (X, 1*H+1*G) fill lightblue
    P1: box "Phase 1: install.sh  (root)" fill 0xfff0cc width 3.2 height 0.45 rad 5px
    text "binary · nats user · dirs · TLS cert · systemd unit" small at P1.s + (0, -0.18)
    arrow from P1.s down 0.28 color 0x888888
    P2: box "Phase 2: bootstrap-creds.sh  (dmestas)" fill 0xd0f0d0 width 3.2 height 0.45 rad 5px
    text "nsc operator · accounts · users · .creds files · operator.env" small at P2.s + (0, -0.18)
    arrow from P2.s down 0.28 color 0x888888
    P3: box "Phase 3: start.sh  (root)" fill 0xd0eeff width 3.2 height 0.45 rad 5px
  text "run as root" small at last box.s + (0,0.1)
box "Phase 4: provision-streams.sh" wid W ht H rad 6px at (X, 0) fill lightcyan
  text "run as dmestas" small at last box.s + (0,0.1)

arrow from (X, 3*H+3*G - H/2) down G+0.05
    text "render config · start service · copy account JWTs" small at P3.s + (0, -0.18)
    arrow from P3.s down 0.28 color 0x888888
arrow from (X, 2*H+2*G - H/2) down G+0.05
arrow from (X, 1*H+1*G - H/2) down G+0.05

    P4: box "Phase 4: provision-streams.sh  (dmestas)" fill 0xd0f0ff width 3.2 height 0.45 rad 5px
# Details on right
DX = 2.2
text "Download binary, create nats user, dirs," at (DX+1.8, 3*H+3*G+0.05) small ljust
text "issue TLS cert (acme.sh DNS-01), install systemd unit" at (DX+1.8, 3*H+3*G-0.18) small ljust
text "Create operator/accounts/users via nsc," at (DX+1.8, 2*H+2*G+0.05) small ljust
text "generate .creds files, write operator.env" at (DX+1.8, 2*H+2*G-0.18) small ljust
text "Render config, start nats.service," at (DX+1.8, 1*H+1*G+0.05) small ljust
text "copy account JWTs to resolver dir" at (DX+1.8, 1*H+1*G-0.18) small ljust
text "Create EDGESYNC and GO_LIBFOSSIL" at (DX+1.8, 0+0.05) small ljust
    text "create EDGESYNC and GO_LIBFOSSIL streams" small at P4.s + (0, -0.18)
text "JetStream streams" at (DX+1.8, 0-0.18) small ljust
```

### Running a fresh install

```sh
# On the server, as dmestas:
cd ~/Projects/infra
︙