Firehose phase 3: relay-lists hose (kind 10002) — retires the last legacy kind#12
Merged
Merged
Conversation
Last subscription kind migrated to a hose; the legacy raw-upsert fallback is now empty. - src/hoses/relays.js: shared relay-directory helpers (canonicalizeRelayUrl, harvestRelays via $setOnInsert, ensureRelayDirectoryIndex). Follows hose refactored onto them (its relayUrlsFrom now returns raw content-map keys; canonicalize/dedup/filter is harvestRelays' job). - src/hoses/relaylists.js (kind 10002): verify signature, latest-wins upsert of the raw event into relay_lists (the shape diddoc reads for service entries), harvest r-tag URLs into the relays directory (canonical source, feeds the phase-4 prober). Owns pubkey + created_at indexes. - src/indexer.js: register the hose; LEGACY_KINDS = []. planIngest now takes the legacy-kind set as a parameter so the plan tests are self-contained. - tests: relaylists-hose tests (verify / r-tag extraction); plan tests reworked to inject legacy kinds; follows test updated for the shared helper. npm test 40/40. Closes #11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #11. Phase 3 of the firehose rework (after #7, #9). With this, every event kind is hose-owned and the legacy raw-upsert fallback is empty — our indexer fully owns ingestion.
What
src/hoses/relays.js— shared relay-directory helpers:canonicalizeRelayUrl,harvestRelays(db, urls)($setOnInsert, never clobbers health metrics),ensureRelayDirectoryIndex. The follows hose is refactored onto them (itsrelayUrlsFromnow just returns the raw content-map keys; canonicalize/dedup/filter isharvestRelays' job).src/hoses/relaylists.js(kind 10002):rtags).relay_lists— the shapediddocreads to build the DID document'sservice/Relayentries.pubkey+created_atindexes.r-tag URLs into therelaysdirectory — the canonical relay-URL source, which the phase-4 health prober will check. (The old relayhose did neither verification nor discovery.)src/indexer.js— register the hose;LEGACY_KINDS = [].planIngestnow takes the legacy-kind set as a parameter, so the plan tests are self-contained and stop needing edits each phase.Verification
npm test— 40/40 (new relay-lists tests: verify / wrong kind / tampered / malformed /r-tag extraction; plan tests reworked to inject legacy kinds; follows test updated for the shared helper).r-tag URLs harvested as{relay}only, a pre-existing relay'sonline/uptimeleft untouched, andbuildDidDocumentresolves bothserviceentries from the stored event.hoses: [profiles, follows, relaylists], kind-10002 events ingested (relay_lists+395,relays+103 harvested) into a scratch db.Out of scope
Active relay-health prober (kind-less; unfreezes
/relays) — phase 4 · deploy + retire the legacy server hoses — phase 5.