Skip to content

Commit 373ce79

Browse files
webrtc + tunnel ports, compose test: every plugin on one server (39/39)
- webrtc/: full three-dialect parity, zero imports (activate(api) alone), fixes core's auth race. tunnel/: full parity, single-prefix deviation (control at {prefix}/connect). Both by subagents, independently verified. - compose.test.js: ONE JSS from npm runs all six plugins from pure config, each surface exercised over the wire, pods+WAC intact beside them. - Two bugs the dogfooding surfaced (NOTES): generic-basename id collision (<name>/plugin.js all reduce to 'plugin' — derivation should fall back to parent dir), and dotted prefixes failing the ws upgrade (core reserves /.terminal etc). README results table; findings consolidated.
1 parent 2b91a26 commit 373ce79

8 files changed

Lines changed: 478 additions & 108 deletions

File tree

NOTES.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,26 @@ crisp line #564 needed:
5757
- pipeline-modifying features → core (or a future, separately-gated hooks
5858
capability)
5959

60+
## Bugs the composition surfaced (this repo's own dogfooding)
61+
62+
- **Generic-basename id collision** — every port follows `<name>/plugin.js`,
63+
so all six modules derive the id `plugin` and the loader's duplicate-id
64+
guard (added in PR #589 review) refuses to boot until each entry gets an
65+
explicit `id`. The guard is correct; the *derivation* is weak. Fix
66+
candidate: when the basename is generic (`plugin`, `index`), derive from
67+
the parent directory (`relay/plugin.js``relay`). Small, backward-
68+
compatible, removes the most common footgun. **Filed-worthy.**
69+
- **Dotted prefixes fail the ws upgrade** — a plugin mounted at
70+
`/.terminal` cannot accept WebSocket connections (immediate upgrade
71+
error), while `/terminal` and even `/.notifications` HTTP work. Core
72+
reserves specific dotted paths (`/.terminal`, `/.webrtc`) for its own
73+
built-ins in the WAC-skip list; a plugin claiming a dotted prefix
74+
collides with host-level dotfile/route handling in a way plain prefixes
75+
don't. Consequence: plugins should avoid dotted prefixes, or the loader
76+
should validate/reserve them explicitly. (The composition uses
77+
`/terminal`; `/.notifications` HTTP happens to work but wasn't stressed
78+
for upgrades.)
79+
6080
## Smaller notes
6181

6282
- A plugin owns exactly one prefix. Features with scattered paths (core

README.md

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -38,18 +38,24 @@ A plugin directory may import:
3838
without internals, that gap is the finding — document it in the port's
3939
README and NOTES.md, ship the closest honest approximation.
4040

41-
## Layout
41+
## What's here
42+
43+
| Plugin | Ports | Status | Notable |
44+
|---|---|---|---|
45+
| `relay/` | `src/nostr/relay.js` | ✅ parity + | pluginDir persistence core lacks; vendored NIP-01 verify |
46+
| `webrtc/` | `src/webrtc/index.js` | ✅ full parity | **zero imports**`activate(api)` alone sufficed; fixes an auth race core has |
47+
| `terminal/` | `src/terminal/index.js` | ✅ parity + hardened | mandatory access control, minimal child env, confined cwd |
48+
| `tunnel/` | `src/tunnel/` | ✅ parity | one deviation: single-prefix forces `{prefix}/connect` control path |
49+
| `notifications/` | `src/notifications/` | ✅ parity | **the seam-forcer** — WAC via loopback; forces `api.events`, `api.serverInfo` |
50+
| `pay/` | pay mode | 📋 wall-report | pipeline-modifying → **stays core**; draws the #564 line |
51+
52+
39 tests, all green (`npm test`), including `compose.test.js` — every plugin
53+
on one server from pure config. Findings consolidated in [NOTES.md](./NOTES.md).
4254

4355
```
44-
relay/ NIP-01 nostr relay (port of src/nostr/relay.js)
45-
webrtc/ WebRTC signaling rooms (port of src/webrtc/index.js)
46-
terminal/ WebSocket shell — GATED (port of src/terminal/index.js)
47-
tunnel/ reverse tunnel over WebSocket (port of src/tunnel/)
48-
notifications/ pod change notifications (port of src/notifications/ — the seam-forcer)
49-
pay/ HTTP 402 paid routes (port of src/mrc20.js pay mode — wall-report)
50-
compose.test.js ONE server, every plugin, from pure config
51-
serve.js demo composition
52-
NOTES.md findings log: what the api gave us, what it didn't
56+
<name>/plugin.js the port <name>/test.js real-JSS tests <name>/README.md findings
57+
compose.test.js ONE server, every plugin serve.js demo composition
58+
helpers.js test harness (boot JSS from npm) NOTES.md consolidated findings
5359
```
5460

5561
Each directory: `plugin.js` (exports `activate(api)`), `test.js` (boots a

compose.test.js

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
// The money shot: ONE JavaScript Solid Server from npm, EVERY plugin in
2+
// this repo loaded from pure config, each surface exercised over the wire,
3+
// pods + WAC intact beside them all.
4+
5+
import { describe, it, after } from 'node:test';
6+
import assert from 'node:assert';
7+
import fs from 'node:fs';
8+
import os from 'node:os';
9+
import path from 'node:path';
10+
import { fileURLToPath } from 'node:url';
11+
import { WebSocket } from 'ws';
12+
import { probePort, startJss } from './helpers.js';
13+
import { finalizeEvent, generateSecretKey } from './relay/nip01.js';
14+
15+
const __dirname = path.dirname(fileURLToPath(new URL(import.meta.url)));
16+
const at = (p) => path.join(__dirname, p);
17+
18+
function openWs(url, headers) {
19+
const socket = new WebSocket(url, headers ? { headers } : undefined);
20+
return new Promise((resolve, reject) => {
21+
const lines = [];
22+
socket.on('message', (d) => lines.push(String(d)));
23+
socket.on('open', () => resolve({ socket, lines }));
24+
socket.on('error', reject);
25+
});
26+
}
27+
28+
function waitFor(lines, predicate, timeoutMs = 5000) {
29+
return new Promise((resolve, reject) => {
30+
const started = Date.now();
31+
const timer = setInterval(() => {
32+
const hit = lines.find(predicate);
33+
if (hit) { clearInterval(timer); resolve(hit); }
34+
else if (Date.now() - started > timeoutMs) {
35+
clearInterval(timer);
36+
reject(new Error(`timeout; lines: ${JSON.stringify(lines.slice(0, 10))}`));
37+
}
38+
}, 25);
39+
});
40+
}
41+
42+
describe('composition: every plugin on one server', () => {
43+
let jss;
44+
let base;
45+
let wsBase;
46+
47+
after(async () => { if (jss) await jss.close(); });
48+
49+
it('boots pods + idp + six plugins from config', async () => {
50+
const port = await probePort();
51+
base = `http://127.0.0.1:${port}`;
52+
wsBase = `ws://127.0.0.1:${port}`;
53+
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'jss-compose-'));
54+
jss = await startJss({
55+
port,
56+
root,
57+
idp: true,
58+
// Explicit ids: the <name>/plugin.js convention makes every basename
59+
// reduce to 'plugin' — the loader's duplicate-id guard requires ids
60+
// here (finding: derive from the parent dir for generic basenames).
61+
plugins: [
62+
{ id: 'relay', module: at('relay/plugin.js'), prefix: '/relay' },
63+
{ id: 'webrtc', module: at('webrtc/plugin.js'), prefix: '/webrtc' },
64+
{ id: 'terminal', module: at('terminal/plugin.js'), prefix: '/terminal', config: { token: 'compose-secret' } },
65+
{ id: 'tunnel', module: at('tunnel/plugin.js'), prefix: '/tunnel' },
66+
{
67+
id: 'notifications',
68+
module: at('notifications/plugin.js'),
69+
prefix: '/.notifications',
70+
config: { podsRoot: root, baseUrl: base },
71+
},
72+
{ id: 'pay', module: at('pay/plugin.js'), prefix: '/paid', config: { cost: 2, address: 'x' } },
73+
],
74+
});
75+
assert.ok(jss.base);
76+
});
77+
78+
it('relay: signed event round-trips', async () => {
79+
const { socket, lines } = await openWs(`${wsBase}/relay`);
80+
const event = finalizeEvent({ kind: 1, content: 'compose' }, generateSecretKey());
81+
socket.send(JSON.stringify(['EVENT', event]));
82+
await waitFor(lines, (l) => {
83+
const m = JSON.parse(l);
84+
return m[0] === 'OK' && m[1] === event.id && m[2] === true;
85+
});
86+
socket.close();
87+
});
88+
89+
it('webrtc: content-addressed room relays an offer between peers', async () => {
90+
// Anonymous content-addressed dialect (no credentials needed).
91+
const room = 'a'.repeat(64); // hex hash "resource"
92+
const a = await openWs(`${wsBase}/webrtc`);
93+
const b = await openWs(`${wsBase}/webrtc`);
94+
a.socket.send(JSON.stringify({ type: 'announce', resource: room, offers: [] }));
95+
await waitFor(a.lines, (l) => JSON.parse(l).type === 'resource-peers');
96+
b.socket.send(JSON.stringify({
97+
type: 'announce', resource: room,
98+
offers: [{ sdp: 'compose-offer', offer_id: 'o1' }],
99+
}));
100+
await waitFor(a.lines, (l) => {
101+
const m = JSON.parse(l);
102+
return m.type === 'offer' && m.offer_id === 'o1';
103+
});
104+
a.socket.close();
105+
b.socket.close();
106+
});
107+
108+
it('terminal: token-gated shell echoes', async () => {
109+
const socket = new WebSocket(`${wsBase}/terminal?token=compose-secret`);
110+
let buf = '';
111+
socket.on('message', (d) => { buf += String(d); });
112+
await new Promise((resolve, reject) => {
113+
socket.on('open', resolve);
114+
socket.on('error', reject);
115+
});
116+
await new Promise((r) => setTimeout(r, 400)); // let the shell spawn
117+
socket.send('echo compose-ok\n');
118+
await new Promise((resolve, reject) => {
119+
const t = setTimeout(() => reject(new Error(`no echo; got ${JSON.stringify(buf)}`)), 5000);
120+
const iv = setInterval(() => {
121+
if (buf.includes('compose-ok')) { clearTimeout(t); clearInterval(iv); resolve(); }
122+
}, 25);
123+
});
124+
socket.close();
125+
});
126+
127+
it('notifications: pod write fans out to a subscriber', async () => {
128+
fs.writeFileSync(
129+
path.join(jss.root, 'note.txt.acl'),
130+
`@prefix acl: <http://www.w3.org/ns/auth/acl#>.
131+
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
132+
<#public> a acl:Authorization; acl:agentClass foaf:Agent;
133+
acl:accessTo <./note.txt>; acl:mode acl:Read.
134+
`,
135+
);
136+
const { socket, lines } = await openWs(`${wsBase}/.notifications`);
137+
await waitFor(lines, (l) => l === 'protocol solid-0.1');
138+
socket.send(`sub ${base}/note.txt`);
139+
await waitFor(lines, (l) => l === `ack ${base}/note.txt`);
140+
fs.writeFileSync(path.join(jss.root, 'note.txt'), 'hello');
141+
await waitFor(lines, (l) => l === `pub ${base}/note.txt`);
142+
socket.close();
143+
});
144+
145+
it('pay: 402 then paid content', async () => {
146+
let res = await fetch(`${base}/paid/demo`);
147+
assert.strictEqual(res.status, 402);
148+
res = await fetch(`${base}/paid/demo`, { headers: { 'x-payment-proof': 'demo-proof-of-payment' } });
149+
assert.strictEqual(res.status, 200);
150+
});
151+
152+
it('pods still work beside all of it (idp register + WAC)', async () => {
153+
let res = await fetch(`${base}/idp/register`, {
154+
method: 'POST',
155+
headers: { 'content-type': 'application/json' },
156+
body: JSON.stringify({ username: 'composer', password: 'compose-pass', confirmPassword: 'compose-pass' }),
157+
});
158+
assert.ok(res.status < 400, `register: ${res.status}`);
159+
res = await fetch(`${base}/composer/private/x`, { method: 'PUT', body: 'nope' });
160+
assert.ok([401, 403].includes(res.status), `WAC: ${res.status}`);
161+
});
162+
});

serve.js

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -28,17 +28,19 @@ const fastify = createServer({
2828
root: PODS,
2929
idp: true,
3030
idpIssuer: PUBLIC_URL,
31+
// Explicit ids — the <name>/plugin.js convention collides on basename.
3132
plugins: [
32-
{ module: at('relay/plugin.js'), prefix: '/relay' },
33-
{ module: at('webrtc/plugin.js'), prefix: '/webrtc' },
34-
{ module: at('terminal/plugin.js'), prefix: '/.terminal', config: { token: terminalToken } },
35-
{ module: at('tunnel/plugin.js'), prefix: '/tunnel' },
33+
{ id: 'relay', module: at('relay/plugin.js'), prefix: '/relay' },
34+
{ id: 'webrtc', module: at('webrtc/plugin.js'), prefix: '/webrtc' },
35+
{ id: 'terminal', module: at('terminal/plugin.js'), prefix: '/terminal', config: { token: terminalToken } },
36+
{ id: 'tunnel', module: at('tunnel/plugin.js'), prefix: '/tunnel' },
3637
{
38+
id: 'notifications',
3739
module: at('notifications/plugin.js'),
3840
prefix: '/.notifications',
3941
config: { podsRoot: PODS, baseUrl: PUBLIC_URL, loopbackUrl: `http://127.0.0.1:${PORT}` },
4042
},
41-
{ module: at('pay/plugin.js'), prefix: '/paid', config: { cost: 1, address: 'demo' } },
43+
{ id: 'pay', module: at('pay/plugin.js'), prefix: '/paid', config: { cost: 1, address: 'demo' } },
4244
],
4345
});
4446

@@ -54,7 +56,7 @@ console.log(`jss + every plugin up at ${PUBLIC_URL}`);
5456
console.log(` pods: ${PUBLIC_URL}/idp/register`);
5557
console.log(` relay: ws ${PUBLIC_URL}/relay`);
5658
console.log(` webrtc: ws ${PUBLIC_URL}/webrtc`);
57-
console.log(` terminal: ws ${PUBLIC_URL}/.terminal?token=${terminalToken}`);
59+
console.log(` terminal: ws ${PUBLIC_URL}/terminal?token=${terminalToken}`);
5860
console.log(` tunnel: ws ${PUBLIC_URL}/tunnel`);
5961
console.log(` notifications: ws ${PUBLIC_URL}/.notifications`);
6062
console.log(` paid demo: GET ${PUBLIC_URL}/paid/demo`);

tunnel/README.md

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# tunnel — reverse HTTP tunnel over WebSocket
2+
3+
Out-of-tree port of JSS `src/tunnel/index.js` ("decentralized ngrok",
4+
AGPL-3.0-only): a tunnel client connects over WebSocket, registers a name,
5+
and the pod proxies public HTTP traffic to it.
6+
7+
```js
8+
plugins: [{ module: 'tunnel/plugin.js', prefix: '/tunnel',
9+
config: { requestTimeout: 30000, maxMessageSize: 10485760 } }]
10+
```
11+
12+
- Control socket: `ws://host/tunnel/connect` — authenticated
13+
(`api.auth.getAgent`: Bearer, DPoP, NIP-98, …; `?token=` fallback for
14+
browser clients, #528).
15+
- Public URL: `http://host/tunnel/{name}/path`.
16+
17+
## Protocol (unchanged from core)
18+
19+
```
20+
→ { type: "register", name: "myapp", passthrough?: true }
21+
← { type: "registered", name: "myapp", url: "/tunnel/myapp/", passthrough: bool }
22+
← { type: "request", id, method, path, headers, body?, bodyEncoding? }
23+
→ { type: "response", id, status, headers, body, bodyEncoding? }
24+
← { type: "error", message }
25+
```
26+
27+
Full #530 credential model ported: by default `Cookie`/`Authorization`
28+
are stripped inbound and `Set-Cookie` outbound; a registration may opt in
29+
with `passthrough: true` (strict boolean), which forwards them — except
30+
the relay's own IdP `_session`/`_interaction` cookies and
31+
`Proxy-Authorization`, which never cross. 10MB message cap (#567), 30s
32+
response timeout, name takeover only by the same agent, 308 redirect for
33+
`/tunnel/{name}``/tunnel/{name}/`.
34+
35+
## Deviations from core
36+
37+
- **Single mount prefix.** Core owns two path roots: the control socket
38+
at `/.tunnel` and traffic at `/tunnel/{name}/…`. The loader WAC-exempts
39+
exactly ONE prefix per plugin, so both live under the mount: control at
40+
`{prefix}/connect`, traffic at `{prefix}/{name}/…` — which makes
41+
`connect` a **reserved tunnel name** (registration rejects it).
42+
- **Raw bodies, always base64.** The proxy routes sit in a scoped
43+
pass-through content-type parser (see Findings / JSS #583), so
44+
`request.body` is the untouched Buffer and every forwarded body is
45+
`bodyEncoding: "base64"`. Core re-serializes parsed JSON bodies
46+
(minifying/reordering them); this port is byte-exact — within protocol,
47+
since clients must already handle both encodings.
48+
- **`requestTimeout` / `maxMessageSize` are config knobs** (core
49+
hardcodes 30s / 10MB).
50+
- **Auth frames are queued, not dropped.** Core `await`s verification
51+
before wiring the message handler, so a `register` sent during a slow
52+
credential check (e.g. first did:nostr resolution) is lost. Here every
53+
message awaits the shared auth promise — ordering preserved, nothing
54+
dropped.
55+
- Agents are keyed on `api.auth.getAgent`'s id string (WebID **or**
56+
`did:nostr:` DID) rather than core's `webId` — same semantics, wider
57+
domain.
58+
59+
## Findings
60+
61+
- **The one-prefix rule forced the only real protocol change.** A plugin
62+
cannot own both `/.tunnel` and `/tunnel/`; folding the control socket
63+
into the mount (`{prefix}/connect`) cost a reserved name. Fine here,
64+
but any port whose core feature spans multiple path roots will hit
65+
this seam — plugins may want `prefix: [a, b]` or an
66+
`api.appPaths.add(path)` escape hatch.
67+
- **Fastify body parsing vs. raw proxying (#583) — the known pattern
68+
works.** `api.fastify` is a real scope: `register(async (scope) => {
69+
scope.removeAllContentTypeParsers(); scope.addContentTypeParser('*',
70+
{ parseAs: 'buffer' }, …) })` confines raw-body handling to the proxy
71+
routes without touching the host's parsers. Ergonomic enough, but every
72+
proxying plugin will re-discover it; an `api`-level "give me raw
73+
bodies under this route" helper would close the gap.
74+
- **`api.ws.route` needed no wildcard.** Core's tunnel already
75+
multiplexes all traffic over one control socket, so a single exact ws
76+
path sufficed; wildcard ws routes were never needed. (They'd be needed
77+
by a port whose *clients* connect to per-resource ws paths — the
78+
notifications port is the one that will test that.)
79+
- **No way to pass ws server options.** The loader registers
80+
`@fastify/websocket` itself with defaults, so a plugin can't set
81+
`maxPayload`; like core, the 10MB cap (#567) is enforced app-level
82+
after the frame is already in memory. A `ws.route(path, handler,
83+
{ maxPayload })` passthrough would let the transport reject oversized
84+
frames early.
85+
- **`api.auth.getAgent` covered the gate 1:1** — including the `?token=`
86+
browser fallback (the plugin just copies the query param into the
87+
Authorization header before calling it). No internal imports needed
88+
anywhere in this port; the only import in plugin.js is `node:crypto`.
89+
- **Testing an auth-gated plugin without an IdP:** NIP-98 is the one
90+
credential a test can self-mint (schnorr event over the control URL,
91+
~20 lines with `@noble/curves`); the agent falls back to
92+
`did:nostr:<pubkey>`. First verification may consult the external
93+
did:nostr resolver (timeout-bounded, then cached per pubkey) — the
94+
test budgets 15s for the first ack. A documented "test credential"
95+
seam in the host would make plugin test suites hermetic.
96+
97+
## Test
98+
99+
```bash
100+
node --test --test-concurrency=1 tunnel/test.js
101+
```
102+
103+
10 tests: auth rejection, end-to-end GET through a real local target
104+
(headers both ways, query string), byte-exact POST round-trip, #530
105+
default-strip + passthrough (incl. relay-cookie filtering), reserved and
106+
invalid names, 502 unknown tunnel, 308 trailing-slash redirect, in-flight
107+
disconnect → 502 (and 502 after), unanswered request → 504.

tunnel/plugin.js

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -310,7 +310,9 @@ export async function activate(api) {
310310
return reply.code(502).send({ error: 'Bad Gateway', message: 'Tunnel not connected' });
311311
}
312312

313-
return reply.redirect(308, `${prefix}/${name}/`);
313+
// (new-signature redirect; core's redirect(308, url) form is
314+
// deprecated on the Fastify 4 the host ships)
315+
return reply.redirect(`${prefix}/${name}/`, 308);
314316
});
315317
});
316318

0 commit comments

Comments
 (0)