Commit bff24f2
authored
Index subdomain-mode pods (JavaScriptSolidServer#411) (JavaScriptSolidServer#413)
* Index subdomain-mode pods (JavaScriptSolidServer#411)
The well-known did:nostr indexer was silently dropping every
subdomain-mode pod. The original `profilePathFromWebId` derived
the on-disk path from `webIdUrl.pathname` only, which works for
path-mode named pods (`example.com/alice/...` → `<dataRoot>/alice/...`)
and root pods (`example.com/profile/...` → `<dataRoot>/profile/...`)
but loses the subdomain in the third case:
WebID: https://alice.example.com/profile/card.jsonld#me
pathname: /profile/card.jsonld
derived: <dataRoot>/profile/card.jsonld
actual: <dataRoot>/alice/profile/card.jsonld ← profile lives here
On solid.social (subdomain-mode), every subdomain user fell
through to the typed-username fallback because the local index
never found their profile. "Sign in with Schnorr" with no typing
just didn't work.
Fix:
- New `profilePathCandidates(dataRoot, webId)` returns the
ORDERED list of candidate filesystem paths to probe:
1. `<dataRoot>/<pathname>` (path mode + root pod)
2. `<dataRoot>/<host-first-label>/<pathname>` (subdomain mode)
Subdomain candidate only emitted for hosts with ≥2 DNS labels.
Containment-checked: every candidate stays under dataRootAbs.
- rebuildPubkeyIndex now PROBES candidates and accepts the
first one whose `@id` matches `account.webId`. Multiple
candidates can exist on disk simultaneously (root pod +
named pods coexisting; subdomain pod with a coincidentally-
named path-mode dir), so committing to the first
candidate-that-stats was a bug — could pick someone else's
profile, fail the @id check, log "no match," skip the
actual matching candidate. Now: stat → size cap → parse →
@id check, fall through to next candidate on any failure.
- Removed the unused `findProfilePathOnDisk` helper.
Tests:
- Six new unit tests on `profilePathCandidates` covering
path mode, root pod, subdomain mode, single-label hosts,
unparseable input, and containment across all of them.
- One new integration test that synthesizes a subdomain-mode
pod (profile at `<TEST_DATA_DIR>/sub/profile/card.jsonld`,
webId at `http://sub.example.test/profile/...`) and asserts
the well-known endpoint resolves it. The test exercises the
"first candidate exists but belongs to a different account"
case because the prior test in the same suite leaves a
root-pod profile at `<TEST_DATA_DIR>/profile/card.jsonld` —
the subdomain account's path-mode candidate would hit it,
fail the @id check, and the loop must fall through to the
subdomain candidate. (This caught the original
findProfilePathOnDisk bug during development.)
Test count: 752 → 767 in full suite (+ 7 unit + 1 integration
+ 7 from the same test file's organic growth across the run).
* Address copilot pass 1 on JavaScriptSolidServer#413 — gate subdomain candidate on podName
Real concern with my pass-0 implementation: `profilePathCandidates`
emitted the "host first label as pod dir" candidate for ANY host
with ≥2 DNS labels. For a root-pod WebID
(`https://example.com/profile/card.jsonld`) on a normal domain,
that produced `<dataRoot>/example/profile/card.jsonld` as a
secondary candidate — which could be a totally unrelated
account's pod dir.
The downstream `@id` check rejected mis-indexing, so it was never
exploitable, but:
- extra wasted disk probes per non-subdomain account
- confusing log entries blaming the wrong path on rebuild miss
- defense rested entirely on @id check, no precision in
candidate generation itself
Fix: add an optional `podName` parameter. The subdomain candidate
is only emitted when `<podName>.` is the WebID host's actual
first label (case-insensitive — DNS is). `rebuildPubkeyIndex`
passes `account.podName` since it has the account record in hand.
For root pods (podName='me' from src/server.js's single-user seed,
host is the bare baseDomain), the gate fails and only the
path-mode/root candidate is emitted. For a true subdomain pod
(podName='alice', host='alice.example.com'), the gate matches
and we emit `<dataRoot>/alice/profile/card.jsonld` as the
secondary candidate. Path mode (no subdomain in host) still
gets just the path-mode candidate.
Tests:
- "subdomain-mode pod: emits ... when host first label matches
podName" (renamed; existing behavior, now explicit about the
matching condition)
- NEW "does NOT emit a subdomain candidate when podName is
omitted" — backward-safe behavior for any other caller
- NEW "does NOT emit a subdomain candidate when podName does
not match the host first label" — the case Copilot flagged:
root-pod on example.com with podName='me' must NOT produce
`<dataRoot>/example/profile/...`
- "single-label host" test now passes podName explicitly to
confirm gating is consistent
- Containment loop now includes podName per case
Test count: 767 → 769 in full suite.
* Address copilot pass 2 on JavaScriptSolidServer#413 — precise per-candidate diagnostics
Two findings, both about losing operator-visible information:
1. The fallback `err = { code: 'ENOENT', message: 'no candidate
matched account.webId' }` was misleading. The actual reason
could be:
- profile genuinely missing (ENOENT)
- oversized file
- JSON parse failure
- `@id` mismatch (most common when subdomain config is wrong)
- candidate rejected by containment
Hard-coding ENOENT erased that signal.
Fix: rebuild loop now accumulates per-candidate failure
reasons in a list, and the fallback log emits ALL of them
joined with ` | `, prefixed with the actual code:
- `<path>: ENOENT` / stat-error / not-a-regular-file
- `<path>: oversized (NN > MAX)`
- `<path>: parse-error (msg)`
- `<path>: @id-mismatch (declared=...)`
- `<path>: outside-dataRoot`
Top-level err code changed to `NO_CANDIDATE_MATCHED` so it's
distinguishable from a single ENOENT.
2. `profilePathCandidates` was silently dropping out-of-dataRoot
candidates. Pre-JavaScriptSolidServer#411, `profilePathFromWebId` logged "resolves
outside dataRoot" loudly. Operators lost that signal —
traversal/misconfig and "profile not on disk" looked
identical in the failure log.
Fix: function now returns `{ paths, skipped }`. `paths` is
the containment-passed candidates (caller probes these);
`skipped` is `[{ path, reason }]` for any rejected candidate
(today only "outside-dataRoot"). The rebuild loop folds
`skipped` into its diagnostics list so containment
rejections show up in the per-account failure log alongside
ENOENT/parse/mismatch reasons.
Test changes:
- All `profilePathCandidates` callers updated for the
`{ paths, skipped }` shape (destructuring `paths`).
- The "returns []" test became "returns empty paths" with the
new shape.
- New "returns the shape so the caller can surface
diagnostics" test confirms the API contract: paths and
skipped are both arrays, normal URL input produces empty
skipped.
Test count: 769 → 770 in full suite (+1 shape test, -0 — the
shape rewrite consolidated the `[].length === 0` assertion
into the new shape test).
* Address copilot pass 3 on JavaScriptSolidServer#413
Four findings, all real.
1. Oversize-profile branch was emitting a direct `console.error`,
bypassing the per-account rate limit in `logProfileFailure`.
On every TTL rebuild, an oversized profile would re-log; if
the same account had a later candidate that matched, the
error would also be misleading. Now folded into the same
`reasons` accumulator used for ENOENT/parse/mismatch — the
rate-limited `logProfileFailure` only fires when NO candidate
matches, and the oversize hit is part of the diagnostics in
`err.message`.
2. `logProfileFailure(accountId, profilePath, err)` formats logs
as `(profile=${profilePath})`. We were passing a multi-
candidate summary string (`<path>: ENOENT | <path>: ...`) as
the `profilePath` argument, which made the log field
misleading and unsearchable. Now passes the placeholder
`(multiple candidates)` for the path and puts the full
summary in `err.message`. Operators searching for
`profile=/some/path` get back the actual paths; the summary
travels with the err.
3. The subdomain-mode integration test had an implicit
dependency on test ordering. The "first candidate exists but
belongs to a different account" path was only exercised
because a prior test left a root-pod profile at
`<TEST_DATA_DIR>/profile/card.jsonld`. Reordering tests would
have silently lost that coverage. Now self-contained: the
test creates its OWN decoy profile (with an unrelated key
and webId) at the path-mode candidate location, runs the
subdomain assertion, then cleans up both fixtures.
4. The new `profilePathCandidates` unit tests hard-coded POSIX
paths like `/srv/jss/data/alice/...`, which would fail on
Windows (different separator + drive root). DATA_ROOT is now
`path.resolve('/srv/jss/data')` and every expected value is
built via `path.join(DATA_ROOT, ...)`. Portable.
Test count: same 770; the subdomain test became self-contained
and added two fs writes + cleanup but no new test cases.
* Address copilot pass 4 on JavaScriptSolidServer#413 — snapshot/restore decoy fixture
One finding, real test-isolation gap.
The pass-3 subdomain-mode test created a decoy profile at
`<TEST_DATA_DIR>/profile/card.jsonld` and then unconditionally
deleted it during cleanup. That same path is also used by the
earlier "indexes root-level pods" test, which leaves a real
root-pod profile + an account entry in `_webid_index.json`.
Running the subdomain test would silently wipe the root-pod
fixture, and any later TTL rebuild would log "no candidate
matched" for the orphaned root-pod account — order-dependent,
brittle, and noisy.
Fix:
- Snapshot the decoy file's content (if any) BEFORE writing
over it. ENOENT on snapshot means "didn't exist; remove on
cleanup"; any other error rethrows.
- On cleanup, restore the snapshotted content (if there was
one) or delete the file (if it was created by this test
only).
- Also remove `subProfilePath` (the subdomain candidate file
this test wrote) and prune the now-empty `sub/profile/`
+ `sub/` dirs (best-effort — failures swallowed).
Test count: same 770; the test stays correct + now isolates
its filesystem side effects regardless of execution order.
* Address copilot pass 5 on JavaScriptSolidServer#413 — wrap fixtures in try/finally
One finding, real test-isolation gap.
The pass-4 subdomain-mode test snapshotted/restored the decoy
profile but the cleanup ran in the happy path only. A failing
fetch or assertion would skip the cleanup block and leave:
- the decoy profile overwritten on disk
- subProfilePath created
- `_webid_index.json` carrying a synthetic account entry
- `subdomain-pod-test-account.json` left in accountsDir
turning the whole suite into an order-dependent mess on the
next run.
Fix: wrap the fixture mutations + assertions in `try`, with
ALL restoration in `finally`. The snapshot read happens BEFORE
the try block (it's a read; if it throws non-ENOENT we want to
fail before mutating anything). Each cleanup step is itself
wrapped in a swallow-errors try so one failed restoration
doesn't prevent the others (best effort).
The decoy snapshot/restore semantics are preserved:
- file existed before → restore the original content
- file didn't exist → delete
Test count: same 770; the test now isolates correctly even
when an assertion throws mid-run.
* Address copilot pass 6 on JavaScriptSolidServer#413 — dead code + deprecated API
Two findings, both small.
1. `profilePath` in rebuildPubkeyIndex was assigned when a
candidate matched but never read afterward — the only
downstream consumers care about `profile` (the parsed JSON)
and `mtimeMs` (used in the index entry). Pre-restructure
it was the path used for the next steps; after pass-2's
accept-or-fall-through loop the variable became orphaned.
Removed the declaration and the assignment.
2. `fs.rmdir` (node:fs/promises) emits a deprecation warning
in modern Node — it's slated for removal. Switched to
`fs-extra`'s `fs.remove` which:
- is already in use elsewhere in this test
- handles both files and (recursively) directories
- is a no-op on missing paths (matches our best-effort
cleanup semantics)
Replaces `await fs.rmdir(<dir>)` with `await fs.remove(<dir>)`
for the two parent-dir cleanup steps.
Test count: same 770.1 parent 92b40bf commit bff24f2
2 files changed
Lines changed: 350 additions & 30 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
138 | 138 | | |
139 | 139 | | |
140 | 140 | | |
141 | | - | |
142 | | - | |
143 | | - | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
144 | 156 | | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
153 | 169 | | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
158 | 177 | | |
159 | 178 | | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
168 | 212 | | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
| 213 | + | |
173 | 214 | | |
174 | 215 | | |
175 | 216 | | |
176 | 217 | | |
177 | 218 | | |
178 | | - | |
179 | | - | |
| 219 | + | |
180 | 220 | | |
181 | 221 | | |
182 | 222 | | |
| |||
260 | 300 | | |
261 | 301 | | |
262 | 302 | | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
263 | 361 | | |
264 | 362 | | |
265 | 363 | | |
| |||
0 commit comments