feat(relay): per-community usage metrics - #1723
Conversation
Emit per-community usage gauges and counter tags to Datadog: - New usage poller task (BUZZ_USAGE_METRICS_INTERVAL_SECS, default 60, min 5): DB-polled gauges for users, channels, messages, relay members, workflows, git repos, DAU/WAU/MAU, and active channels, all grouped by community; plus in-memory snapshot gauges for ws_connections, users_online, and subscriptions per community - Write-path community tag: events_received/stored, ws_connections, media_uploads, workflow_runs now carry a 'community' label - New counters: buzz_users_created_total and buzz_channels_created_total tagged by community, incremented only on actual first insert/creation - Fix kind 44200 missing from bounded_kind_label (was collapsing to 'other') ensure_user() now returns Result<bool> (true = new insert, false = conflict DO NOTHING) so callers can count new users without a separate query. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
F1: Zero-fill buzz_community_channels, buzz_community_relay_members, and
buzz_community_workflows across bounded label domains so counts going to
zero emit 0 instead of retaining stale last-nonzero values. buzz_community_
users, messages, git_repos, active_users, active_channels, ws_connections,
users_online, and subscriptions were already zero-filled. Remove the now-
unused host() closure.
Add two regression tests:
- subscription.rs: per_community_subscriptions_drops_to_zero_when_all_
subs_removed (in-memory gauge)
- usage.rs: test_channel_counts_drops_to_zero_after_last_channel_deleted
(DB gauge)
F2: Default interval 60s -> 300s (env override unchanged, min 5s). Add
comment naming rollup-table escape hatch at event-rollup queries.
F3: Increment buzz_channels_created_total{type="dm"} at open_dm call
sites in command_executor.rs (two DM paths) and moderation_notices.rs.
F4: Run cargo fmt; split long assertion in subscription.rs per rustfmt.
F5: Fix test_user_counts_scoped_per_community: original insert_user(is_
agent=true) inserted the agent's owner as a separate human, making the
assertion wrong (3 humans, not 2). Rewrite to share owner pubkey so agent
and human count correctly. All 7 ignored usage tests pass against local PG.
F6: FROM git_repo_names (not git_repos which does not exist). Add test
test_git_repo_counts_scoped_per_community covering the corrected relation.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…regates The module header claimed all queries use indexed columns with no full-table scans. That is false for message_counts, active_user_counts, and active_channel_counts, which are exact event-table aggregates (accepted tradeoff from Pass-1 review F2). Split the doc to accurately describe the two query classes and call out the rollup-table escape hatch. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
|
Reviewed at head Tenant safety (vs #1321) — clean
Useful / non-vacuous — yes
The real concern: cardinality × cost at thousands of communitiesPer community the gauges emit ~25 series (users 2 + channels 4 + messages 1 + members 3 + workflows 3 + repos 1 + active_users 6 + active_channels 2 + ws/online/subs 3). Datadog counts unique tag combos including the pod host tag, so:
Zero-fill makes this worse in one specific way: it guarantees the full product of series always exists — no sparsity savings for dormant communities. That's the correct choice for gauge semantics, but it means cost scales with community count, not activity. Recommendation (can be follow-up, but decide before running on a big relay): an env knob to control the per-community family — e.g. Secondary waste: every pod runs identical DB rollupsThe DB-derived gauges are the same value on every pod (hence the Nits
Bottom lineSafe to merge with respect to #1321 — no fence crossings, no client-mintable labels, read-only. Metrics are well-designed and non-vacuous. Before this runs on a relay with thousands of communities, add (or consciously decline) a cardinality control; that's the only thing standing between "useful" and "expensive." |
C2: add unknown bucket to active_user_counts
- Old SQL: LEFT JOIN + FILTER (WHERE u.agent_owner_pubkey IS NULL) counted
pubkeys with no users row as human (NULL passes IS NULL check).
- Fix: three-way FILTER on u.pubkey IS NOT NULL/IS NULL — profileless
posters and agents with missing rows now land in a new 'unknown' gauge
type rather than inflating the human count.
- CommunityActiveUsers gains an 'unknown: i64' field; callers updated to
emit buzz_community_active_users{type=unknown}.
- New ignored PG test: test_active_user_counts_unknown_bucket_for_profileless_poster.
C3: rename buzz_community_users_online → buzz_community_users_online_pod
- The gauge is pod-local (distinct within a pod, not fleet-distinct).
The _pod suffix makes the aggregation semantics explicit at the metric name
level so dashboard authors don't sum and expect global-distinct counts.
- Added comment explaining fleet-wide query pattern (sum across pods).
C4: collect-before-publish in run_usage_metrics_tick
- All DB .await? calls are now grouped at the top of the function.
If any query fails the function returns early with no metrics emitted,
preventing a mixed fresh/stale snapshot. In-memory snapshots
(infallible) are taken once before the publish phase.
C5: document double-count analysis in handle_create_group
- No code change needed: the Err(_) fallback path only fires when the
prior ingest DB lookup returns an error, not when ingest succeeded.
Added a comment explaining this so the invariant is auditable.
N1: fix PR body — default interval was documented as 60s, is actually 300s.
N2: jitter first tick + MissedTickBehavior::Skip
- PID-seeded Fibonacci hash gives each pod a start delay in
[0, interval_secs), preventing a rolling-deploy thundering herd.
- MissedTickBehavior::Skip prevents catch-up burst if a tick runs long.
N3: warn! on unmatched enum values in filter_map
- channel_type, relay_member role, and workflow status filter_maps now
emit a tracing warn! for unrecognised values instead of silently
dropping them, making schema drift immediately observable in logs.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…nity knob)
C1 — cardinality fix for buzz_events_received_total and buzz_events_stored_total:
- bounded_kind_label passes through all 10k values in 20000..=29999
(client-controlled ephemeral range). Crossing kind × community with
per-community tags produces up to millions of series — this PR had
added 'community' to both counters.
- Fix: revert both counters to {kind}-only (fleet-wide, pre-PR shape).
- Add buzz_community_events_received_total{community} — a new counter
with community-only label for per-community throughput graphs.
Never cross kind × community.
K1 — BUZZ_USAGE_METRICS_PER_COMMUNITY cardinality knob:
- New PerCommunityMode enum parsed from env at startup.
- Three modes: 'all' (default), 'off' (fleet totals only), 'top:<k>'
(per-community series for the k communities with most messages —
reuses message_rows already computed each tick, no extra query).
- Fleet-wide totals (buzz_total_*) always emit regardless of mode,
preserving aggregate visibility at zero extra cardinality cost:
buzz_total_users, buzz_total_channels, buzz_total_messages,
buzz_total_relay_members, buzz_total_workflows, buzz_total_git_repos,
buzz_total_active_users, buzz_total_active_channels,
buzz_total_ws_connections, buzz_total_users_online_pod,
buzz_total_subscriptions.
- Malformed 'top:<k>' values warn! and fall back to 'all'.
PR description updated to reflect final state (C1-K1 + earlier C2-N3).
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
F1: Replace PID-derived first-tick jitter with rand::random::<u64>() % interval_secs. In K8s the relay is typically PID 1 in every pod, so the old hash of std::process::id() produced the same delay on every pod — defeating the thundering-herd mitigation entirely. F2: Remove PerCommunityMode::Top(usize) from the BUZZ_USAGE_METRICS_PER_COMMUNITY knob. As designed, top:<k> was not a real cardinality bound: HashMap iteration order made community selection non-deterministic across pods (union semantics, not k), and the zero-fill design provides no eviction path for communities that leave the active set, so their last gauge values persist until pod restart. The knob now accepts only all|off. A proper top:<k> implementation (stable tie-breaking, gauge idle-timeout lifecycle) is tracked as a fast-follow PR. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
|
Re-reviewed at head My asks — all resolved
Wren's blockers — all resolved
What's knowingly deferred (fine, but say it out loud at deploy time)
None of these three block merge; 1–2 belong in operator docs before anyone points a 5k-community relay at this. Nice, disciplined revision — every fix came with either a test or a documented rationale. |
## Buzz Relay release v0.2.1 ### Changes since relay-v0.2.0: - fix(sdk): preserve self-mention p tags in message and forum event builders ([#4975](#4975)) ([`78c87ae20e`](78c87ae)) - feat(desktop): adding rich link previews to messages ([#3818](#3818)) ([`1922d49cb2`](1922d49)) - feat(relay): accept kind:30179 private managed-agent events at ingest ([#5133](#5133)) ([`ad923353a2`](ad92335)) - fix(media): require authenticated reads ([#4610](#4610)) ([`769ac70b74`](769ac70)) - feat(identity): recover desktop identity from a signed-in phone ([#4845](#4845)) ([`6eb65919f1`](6eb6591)) - ci: prove the relay-driven mesh lifecycle — discover, join, infer, deny — with real nodes ([#3862](#3862)) ([`38bf642fcf`](38bf642)) - relay: fuzz WebSocket 1012 restart-close timing on graceful drain (BUZZ_DRAIN_JITTER_MS) ([#4542](#4542)) ([`e14fff74d0`](e14fff7)) - fix(reactions): support max-length custom emoji ([#3833](#3833)) ([`2ea9385015`](2ea9385)) - fix(channels): restrict private-channel invitations ([#4612](#4612)) ([`efe1893dd3`](efe1893)) - fix(workflow): bind trigger author to the signed event ([#4607](#4607)) ([`885bed35ee`](885bed3)) - fix(git): revoke access for banned relay members ([#4608](#4608)) ([`997b8caaa4`](997b8ca)) - Define private managed agent wire protocol ([#4593](#4593)) ([`067c085f37`](067c085)) - perf(relay): index channel-id lookups and skip trace-only reads ([#4647](#4647)) ([`bc9e6528a7`](bc9e652)) - Polish mobile inbox and media flows ([#4512](#4512)) ([`feccf4eabc`](feccf4e)) - fix(git): allow deleting the default branch ([#4297](#4297)) ([`fc598f5f8d`](fc598f5)) - feat(projects): add buzz projects CLI commands (NIP-MP kind:30621) ([#4020](#4020)) ([`b7bb15122e`](b7bb151)) - perf(relay): serve relay-membership checks from the read replica ([#4124](#4124)) ([`ac4fa13b8e`](ac4fa13)) - fix(relay): allow open relays to set their NIP-11 workspace icon (kind:9033) ([#3998](#3998)) ([`5765fc74b7`](5765fc7)) - feat(relay): accept kind:30621 multi-repo projects at ingest ([#3171](#3171)) ([`cb9701cd30`](cb9701c)) - feat(relay): raise hosted community limit to five ([#3829](#3829)) ([`10d5a26414`](10d5a26)) - fix(relay): align NIP-11 max_limit with REQ ceiling ([#3635](#3635)) ([`23f0c26b1c`](23f0c26)) - feat(relay): gate kind 30178 team-catalog reads behind the shared tag ([#3358](#3358)) ([`114d40d9d3`](114d40d)) - fix(db): isolate usage metrics advisory-lock test on scratch DB ([#3670](#3670)) ([`dba97eecd9`](dba97ee)) - perf(presence): reduce heartbeat frequency ([#3783](#3783)) ([`bf139e8d0b`](bf139e8)) - feat(mesh): upgrade embedded mesh to v0.74 and harden shared compute (split 1/2 of #3467) ([#3741](#3741)) ([`4933672eb4`](4933672)) - feat(replica): portable heartbeat-token fence with snapshot-local reader routing ([#3268](#3268)) ([`63496cc1d4`](63496cc)) - fix(git): channel binding tooling + author remediation for unbound repos ([#3626](#3626)) ([`788b3c002b`](788b3c0)) - feat: configure S3 URL addressing style ([#3400](#3400)) ([`7012d86d52`](7012d86)) - feat(tracing): correlate trace IDs in relay logs ([#3608](#3608)) ([`005b5b819a`](005b5b8)) - fix(relay): avoid subscription lock inversion ([#3413](#3413)) ([`22be8bb351`](22be8bb)) - feat(cli): add users set-status command for NIP-38 profile status ([#3253](#3253)) ([`60158fce3e`](60158fc)) - feat(relay): make Postgres pool size configurable, default 50 ([#3191](#3191)) ([`2ce2d71cc3`](2ce2d71)) - feat(tracing): add datastore tracing plumbing ([#2760](#2760)) ([`e94b9aeda0`](e94b9ae)) - feat(invites): add use-limited invite links ([#3141](#3141)) ([`d500c2d5cf`](d500c2d)) - feat(admin): show reported message content in report detail ([#3149](#3149)) ([`f069a85503`](f069a85)) - resolve findings ([#3150](#3150)) ([`9b0f744804`](9b0f744)) - Revert "fix(cli,relay): resolve agents by verified owner" ([#3168](#3168)) ([`a041e2d21e`](a041e2d)) - fix(cli,relay): resolve agents by verified owner ([#2615](#2615)) ([`c3084b36d9`](c3084b3)) - fix(security): enforce durable community ban on NIP-43 relay-admin kinds 9030-9033 ([#3128](#3128)) ([`e2e0079101`](e2e0079)) - fix(security): authorize kind:9000 role changes in both directions ([#3017](#3017)) ([`00ecf2cac7`](00ecf2c)) - feat(desktop): handle project work from Inbox ([#3117](#3117)) ([`c5c4f390b6`](c5c4f39)) - feat(relay): make per-owner community limit configurable via BUZZ_MAX_COMMUNITIES_PER_OWNER ([#2599](#2599)) ([`2a051a404d`](2a051a4)) - feat(relay): add author-only-unless-shared read gate for kind 30175 ([#2768](#2768)) ([`ab3af82871`](ab3af82)) - fix(core): block IPv6 transition SSRF targets ([#2801](#2801)) ([`c26bf5945d`](c26bf59)) - fix(workflow): bypass system proxies for webhooks ([#2800](#2800)) ([`60a171b19e`](60a171b)) - fix(audit): hash created_at at the precision Postgres stores ([#2638](#2638)) ([`264a56a226`](264a56a)) - feat(desktop): make pull request reviews actionable ([#2510](#2510)) ([`9081ab0ec9`](9081ab0)) - fix(relay): decompress gzip-encoded git smart-HTTP request bodies ([#2670](#2670)) ([`5ca36e7b91`](5ca36e7)) - fix(sharing): preserve agent/team snapshot tEXt chunks through media sanitization ([#2438](#2438)) ([`b096b0a15a`](b096b0a)) - fix(relay): send 1012 restart close to all clients on graceful drain ([#2575](#2575)) ([`1911c69aa2`](1911c69)) - fix(media): sanitize animated image uploads ([#2524](#2524)) ([`8f8f5fa5a4`](8f8f5fa)) - fix(channels): strip leading hash prefixes from names ([#2250](#2250)) ([`d0ab3fdb05`](d0ab3fd)) - feat(relay): make Redis pool size configurable, default 16 ([#2521](#2521)) ([`bcc3e13069`](bcc3e13)) - feat(desktop+acp): spawn a harness per (agent, community) pair at GUI startup — warm sockets, lazy LLM pool ([#2122](#2122)) ([`61cc738ee8`](61cc738)) - feat(media): add S3-truth per-community storage sweep ([#2044](#2044)) ([`bd37a4d584`](bd37a4d)) - feat(relay): log NIP-98 pubkey attribution on HTTP bridge requests ([#2206](#2206)) ([`7e34bee62c`](7e34bee)) - Revert "feat(relay): inventory unreachable Git objects" ([#2275](#2275)) ([`0fb820f9bf`](0fb820f)) - feat(relay): inventory unreachable Git objects ([#2264](#2264)) ([`3afc9dae15`](3afc9da)) - relay: add author_type label to buzz_events_stored_total ([#2243](#2243)) ([`b9f54c43fe`](b9f54c4)) - fix(git): make project branch workflows reliable ([#2213](#2213)) ([`166f27be4b`](166f27b)) - feat(cli): manage repository protection rules ([#2193](#2193)) ([`f94324598d`](f943245)) - feat(cli): add agents archive/unarchive/archived subcommands ([#2173](#2173)) ([`7d7992067b`](7d79920)) - fix(mobile): sanitize Android image uploads ([#2188](#2188)) ([`ee21da90bd`](ee21da9)) - fix(cli): paginate channel directory queries ([#2181](#2181)) ([`03fe19d603`](03fe19d)) - fix(mobile): image upload fails due to unstripped metadata ([#2185](#2185)) ([`37f15b2001`](37f15b2)) - perf(relay): compact Git packs before manifest limits ([#2172](#2172)) ([`80e0ab16b0`](80e0ab1)) - perf(relay): cache Git pack hydration ([#2169](#2169)) ([`a4d82ec722`](a4d82ec)) - fix(relay): bound and observe Git read operations ([#2167](#2167)) ([`5f7c93d9c1`](5f7c93d)) - relay: gate push enqueue on live leases; batch matcher pipeline (T1b/T1a-repair/T2b) ([#2145](#2145)) ([`e43b2d5aac`](e43b2d5)) - relay: add audit logging disable switch ([#2134](#2134)) ([`bf5acabdde`](bf5acab)) - relay: skip TTL deadline bump for known-permanent channels (T1a write-amp) ([#2125](#2125)) ([`2e936d439c`](2e936d4)) - fix(git): carry NIP-OA delegation in auth event ([#2120](#2120)) ([`c12257d57a`](c12257d)) - Route lag-tolerant reads to an optional Postgres read replica ([#2084](#2084)) ([`29c48883d3`](29c4888)) - fix: recover community access visibility ([#2074](#2074)) ([`ca384d082d`](ca384d0)) - feat: proxy feedback-scoped admin attachments ([#2059](#2059)) ([`d7f918e3cb`](d7f918e)) - feat: add read-only deployment moderation dashboard ([#1999](#1999)) ([`68e670e001`](68e670e)) - Bug-bash round 2: table scroll, Goose instructions, workflow mention wake ([#2034](#2034)) ([`64b8fea6dc`](64b8fea)) - Strip media metadata on clients and reject it at the relay ([#2006](#2006)) ([`5cfd69cb0c`](5cfd69c)) - [codex] Hold Git concurrency permits through streaming (BUZZ-SEC-018) ([#1916](#1916)) ([`7baea42abb`](7baea42)) - [codex] Enforce shared relay admission limits (BUZZ-SEC-019) ([#1917](#1917)) ([`73fc0ec6cf`](73fc0ec)) - [codex] Block banned actors from moderation commands (BUZZ-SEC-007) ([#1915](#1915)) ([`caa195ca58`](caa195c)) - [codex] Fix relay WebSocket admission limits ([#1682](#1682)) ([`d3ce971fc7`](d3ce971)) - feat: add invite QR and mobile direct join ([#1957](#1957)) ([`648cbf3610`](648cbf3)) - fix(join-policy): require legal consent on hosted invites ([#1987](#1987)) ([`2e1577f76f`](2e1577f)) - [codex] Prevent actor-tag UI impersonation ([#1931](#1931)) ([`c540ec9678`](c540ec9)) - Scope relay runtime state by community ([#1658](#1658)) ([`d52dedb06f`](d52dedb)) - Apply optional relay join policy across join flows ([#1894](#1894)) ([`6c2d667575`](6c2d667)) - feat(media): require auth for relay media reads ([#1926](#1926)) ([`f308762852`](f308762)) - feat(relay): add community unarchive endpoint ([#1908](#1908)) ([`6b9641db2b`](6b9641d)) - feat(relay): gate Git web GUI separately ([#1901](#1901)) ([`34dc7dec75`](34dc7de)) - mesh: upgrade runtime, enforce membership, add shared compute provider ([#1656](#1656)) ([`54638ff4bb`](54638ff)) - Route Git scratch through configured volume ([#1884](#1884)) ([`2318b3096c`](2318b30)) - feat(relay): gate usage metrics behind stable leader ([#1814](#1814)) ([`59e9821503`](59e9821)) - Relay mesh: cross-pod tunnel + huddle transport (buzz-relay-mesh) ([#1670](#1670)) ([`ccb021d713`](ccb021d)) - feat(push): deliver accepted relay events as wakes ([#1866](#1866)) ([`bffbc5f22c`](bffbc5f)) - fix(db): resolve duplicate migration version ([#1863](#1863)) ([`08ad38a07f`](08ad38a)) - Add private product feedback sidecar ([#1857](#1857)) ([`af190c93e1`](af190c9)) - feat(relay): add durable community archival ([#1834](#1834)) ([`2b15a72675`](2b15a72)) - feat(push): add public APNs gateway ([#1770](#1770)) ([`1c006822e4`](1c00682)) - feat(relay): add atomic community ownership transfer ([#1845](#1845)) ([`52e42ccb9f`](52e42cc)) - Bound NIP-RS retention and search indexing ([#1771](#1771)) ([`1b4703021d`](1b47030)) - Add optional standalone pairing relay to Helm chart ([#1799](#1799)) ([`9b47c8548f`](9b47c85)) - fix(relay): publish membership snapshot on provisioning ([#1761](#1761)) ([`0950d392b7`](0950d39)) - feat(relay): per-community usage metrics ([#1723](#1723)) ([`620822899a`](6208228)) - refactor(desktop): remove vestigial MCP toolsets config ([#1776](#1776)) ([`dfec75b3c0`](dfec75b)) **To release:** merge this PR. The tag and build will happen automatically. Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
## Buzz Relay release v0.2.1 ### Changes since relay-v0.2.0: - fix(sdk): preserve self-mention p tags in message and forum event builders ([block#4975](block#4975)) ([`78c87ae20e`](block@78c87ae)) - feat(desktop): adding rich link previews to messages ([block#3818](block#3818)) ([`1922d49cb2`](block@1922d49)) - feat(relay): accept kind:30179 private managed-agent events at ingest ([block#5133](block#5133)) ([`ad923353a2`](block@ad92335)) - fix(media): require authenticated reads ([block#4610](block#4610)) ([`769ac70b74`](block@769ac70)) - feat(identity): recover desktop identity from a signed-in phone ([block#4845](block#4845)) ([`6eb65919f1`](block@6eb6591)) - ci: prove the relay-driven mesh lifecycle — discover, join, infer, deny — with real nodes ([block#3862](block#3862)) ([`38bf642fcf`](block@38bf642)) - relay: fuzz WebSocket 1012 restart-close timing on graceful drain (BUZZ_DRAIN_JITTER_MS) ([block#4542](block#4542)) ([`e14fff74d0`](block@e14fff7)) - fix(reactions): support max-length custom emoji ([block#3833](block#3833)) ([`2ea9385015`](block@2ea9385)) - fix(channels): restrict private-channel invitations ([block#4612](block#4612)) ([`efe1893dd3`](block@efe1893)) - fix(workflow): bind trigger author to the signed event ([block#4607](block#4607)) ([`885bed35ee`](block@885bed3)) - fix(git): revoke access for banned relay members ([block#4608](block#4608)) ([`997b8caaa4`](block@997b8ca)) - Define private managed agent wire protocol ([block#4593](block#4593)) ([`067c085f37`](block@067c085)) - perf(relay): index channel-id lookups and skip trace-only reads ([block#4647](block#4647)) ([`bc9e6528a7`](block@bc9e652)) - Polish mobile inbox and media flows ([block#4512](block#4512)) ([`feccf4eabc`](block@feccf4e)) - fix(git): allow deleting the default branch ([block#4297](block#4297)) ([`fc598f5f8d`](block@fc598f5)) - feat(projects): add buzz projects CLI commands (NIP-MP kind:30621) ([block#4020](block#4020)) ([`b7bb15122e`](block@b7bb151)) - perf(relay): serve relay-membership checks from the read replica ([block#4124](block#4124)) ([`ac4fa13b8e`](block@ac4fa13)) - fix(relay): allow open relays to set their NIP-11 workspace icon (kind:9033) ([block#3998](block#3998)) ([`5765fc74b7`](block@5765fc7)) - feat(relay): accept kind:30621 multi-repo projects at ingest ([block#3171](block#3171)) ([`cb9701cd30`](block@cb9701c)) - feat(relay): raise hosted community limit to five ([block#3829](block#3829)) ([`10d5a26414`](block@10d5a26)) - fix(relay): align NIP-11 max_limit with REQ ceiling ([block#3635](block#3635)) ([`23f0c26b1c`](block@23f0c26)) - feat(relay): gate kind 30178 team-catalog reads behind the shared tag ([block#3358](block#3358)) ([`114d40d9d3`](block@114d40d)) - fix(db): isolate usage metrics advisory-lock test on scratch DB ([block#3670](block#3670)) ([`dba97eecd9`](block@dba97ee)) - perf(presence): reduce heartbeat frequency ([block#3783](block#3783)) ([`bf139e8d0b`](block@bf139e8)) - feat(mesh): upgrade embedded mesh to v0.74 and harden shared compute (split 1/2 of block#3467) ([block#3741](block#3741)) ([`4933672eb4`](block@4933672)) - feat(replica): portable heartbeat-token fence with snapshot-local reader routing ([block#3268](block#3268)) ([`63496cc1d4`](block@63496cc)) - fix(git): channel binding tooling + author remediation for unbound repos ([block#3626](block#3626)) ([`788b3c002b`](block@788b3c0)) - feat: configure S3 URL addressing style ([block#3400](block#3400)) ([`7012d86d52`](block@7012d86)) - feat(tracing): correlate trace IDs in relay logs ([block#3608](block#3608)) ([`005b5b819a`](block@005b5b8)) - fix(relay): avoid subscription lock inversion ([block#3413](block#3413)) ([`22be8bb351`](block@22be8bb)) - feat(cli): add users set-status command for NIP-38 profile status ([block#3253](block#3253)) ([`60158fce3e`](block@60158fc)) - feat(relay): make Postgres pool size configurable, default 50 ([block#3191](block#3191)) ([`2ce2d71cc3`](block@2ce2d71)) - feat(tracing): add datastore tracing plumbing ([block#2760](block#2760)) ([`e94b9aeda0`](block@e94b9ae)) - feat(invites): add use-limited invite links ([block#3141](block#3141)) ([`d500c2d5cf`](block@d500c2d)) - feat(admin): show reported message content in report detail ([block#3149](block#3149)) ([`f069a85503`](block@f069a85)) - resolve findings ([block#3150](block#3150)) ([`9b0f744804`](block@9b0f744)) - Revert "fix(cli,relay): resolve agents by verified owner" ([block#3168](block#3168)) ([`a041e2d21e`](block@a041e2d)) - fix(cli,relay): resolve agents by verified owner ([block#2615](block#2615)) ([`c3084b36d9`](block@c3084b3)) - fix(security): enforce durable community ban on NIP-43 relay-admin kinds 9030-9033 ([block#3128](block#3128)) ([`e2e0079101`](block@e2e0079)) - fix(security): authorize kind:9000 role changes in both directions ([block#3017](block#3017)) ([`00ecf2cac7`](block@00ecf2c)) - feat(desktop): handle project work from Inbox ([block#3117](block#3117)) ([`c5c4f390b6`](block@c5c4f39)) - feat(relay): make per-owner community limit configurable via BUZZ_MAX_COMMUNITIES_PER_OWNER ([block#2599](block#2599)) ([`2a051a404d`](block@2a051a4)) - feat(relay): add author-only-unless-shared read gate for kind 30175 ([block#2768](block#2768)) ([`ab3af82871`](block@ab3af82)) - fix(core): block IPv6 transition SSRF targets ([block#2801](block#2801)) ([`c26bf5945d`](block@c26bf59)) - fix(workflow): bypass system proxies for webhooks ([block#2800](block#2800)) ([`60a171b19e`](block@60a171b)) - fix(audit): hash created_at at the precision Postgres stores ([block#2638](block#2638)) ([`264a56a226`](block@264a56a)) - feat(desktop): make pull request reviews actionable ([block#2510](block#2510)) ([`9081ab0ec9`](block@9081ab0)) - fix(relay): decompress gzip-encoded git smart-HTTP request bodies ([block#2670](block#2670)) ([`5ca36e7b91`](block@5ca36e7)) - fix(sharing): preserve agent/team snapshot tEXt chunks through media sanitization ([block#2438](block#2438)) ([`b096b0a15a`](block@b096b0a)) - fix(relay): send 1012 restart close to all clients on graceful drain ([block#2575](block#2575)) ([`1911c69aa2`](block@1911c69)) - fix(media): sanitize animated image uploads ([block#2524](block#2524)) ([`8f8f5fa5a4`](block@8f8f5fa)) - fix(channels): strip leading hash prefixes from names ([block#2250](block#2250)) ([`d0ab3fdb05`](block@d0ab3fd)) - feat(relay): make Redis pool size configurable, default 16 ([block#2521](block#2521)) ([`bcc3e13069`](block@bcc3e13)) - feat(desktop+acp): spawn a harness per (agent, community) pair at GUI startup — warm sockets, lazy LLM pool ([block#2122](block#2122)) ([`61cc738ee8`](block@61cc738)) - feat(media): add S3-truth per-community storage sweep ([block#2044](block#2044)) ([`bd37a4d584`](block@bd37a4d)) - feat(relay): log NIP-98 pubkey attribution on HTTP bridge requests ([block#2206](block#2206)) ([`7e34bee62c`](block@7e34bee)) - Revert "feat(relay): inventory unreachable Git objects" ([block#2275](block#2275)) ([`0fb820f9bf`](block@0fb820f)) - feat(relay): inventory unreachable Git objects ([block#2264](block#2264)) ([`3afc9dae15`](block@3afc9da)) - relay: add author_type label to buzz_events_stored_total ([block#2243](block#2243)) ([`b9f54c43fe`](block@b9f54c4)) - fix(git): make project branch workflows reliable ([block#2213](block#2213)) ([`166f27be4b`](block@166f27b)) - feat(cli): manage repository protection rules ([block#2193](block#2193)) ([`f94324598d`](block@f943245)) - feat(cli): add agents archive/unarchive/archived subcommands ([block#2173](block#2173)) ([`7d7992067b`](block@7d79920)) - fix(mobile): sanitize Android image uploads ([block#2188](block#2188)) ([`ee21da90bd`](block@ee21da9)) - fix(cli): paginate channel directory queries ([block#2181](block#2181)) ([`03fe19d603`](block@03fe19d)) - fix(mobile): image upload fails due to unstripped metadata ([block#2185](block#2185)) ([`37f15b2001`](block@37f15b2)) - perf(relay): compact Git packs before manifest limits ([block#2172](block#2172)) ([`80e0ab16b0`](block@80e0ab1)) - perf(relay): cache Git pack hydration ([block#2169](block#2169)) ([`a4d82ec722`](block@a4d82ec)) - fix(relay): bound and observe Git read operations ([block#2167](block#2167)) ([`5f7c93d9c1`](block@5f7c93d)) - relay: gate push enqueue on live leases; batch matcher pipeline (T1b/T1a-repair/T2b) ([block#2145](block#2145)) ([`e43b2d5aac`](block@e43b2d5)) - relay: add audit logging disable switch ([block#2134](block#2134)) ([`bf5acabdde`](block@bf5acab)) - relay: skip TTL deadline bump for known-permanent channels (T1a write-amp) ([block#2125](block#2125)) ([`2e936d439c`](block@2e936d4)) - fix(git): carry NIP-OA delegation in auth event ([block#2120](block#2120)) ([`c12257d57a`](block@c12257d)) - Route lag-tolerant reads to an optional Postgres read replica ([block#2084](block#2084)) ([`29c48883d3`](block@29c4888)) - fix: recover community access visibility ([block#2074](block#2074)) ([`ca384d082d`](block@ca384d0)) - feat: proxy feedback-scoped admin attachments ([block#2059](block#2059)) ([`d7f918e3cb`](block@d7f918e)) - feat: add read-only deployment moderation dashboard ([block#1999](block#1999)) ([`68e670e001`](block@68e670e)) - Bug-bash round 2: table scroll, Goose instructions, workflow mention wake ([block#2034](block#2034)) ([`64b8fea6dc`](block@64b8fea)) - Strip media metadata on clients and reject it at the relay ([block#2006](block#2006)) ([`5cfd69cb0c`](block@5cfd69c)) - [codex] Hold Git concurrency permits through streaming (BUZZ-SEC-018) ([block#1916](block#1916)) ([`7baea42abb`](block@7baea42)) - [codex] Enforce shared relay admission limits (BUZZ-SEC-019) ([block#1917](block#1917)) ([`73fc0ec6cf`](block@73fc0ec)) - [codex] Block banned actors from moderation commands (BUZZ-SEC-007) ([block#1915](block#1915)) ([`caa195ca58`](block@caa195c)) - [codex] Fix relay WebSocket admission limits ([block#1682](block#1682)) ([`d3ce971fc7`](block@d3ce971)) - feat: add invite QR and mobile direct join ([block#1957](block#1957)) ([`648cbf3610`](block@648cbf3)) - fix(join-policy): require legal consent on hosted invites ([block#1987](block#1987)) ([`2e1577f76f`](block@2e1577f)) - [codex] Prevent actor-tag UI impersonation ([block#1931](block#1931)) ([`c540ec9678`](block@c540ec9)) - Scope relay runtime state by community ([block#1658](block#1658)) ([`d52dedb06f`](block@d52dedb)) - Apply optional relay join policy across join flows ([block#1894](block#1894)) ([`6c2d667575`](block@6c2d667)) - feat(media): require auth for relay media reads ([block#1926](block#1926)) ([`f308762852`](block@f308762)) - feat(relay): add community unarchive endpoint ([block#1908](block#1908)) ([`6b9641db2b`](block@6b9641d)) - feat(relay): gate Git web GUI separately ([block#1901](block#1901)) ([`34dc7dec75`](block@34dc7de)) - mesh: upgrade runtime, enforce membership, add shared compute provider ([block#1656](block#1656)) ([`54638ff4bb`](block@54638ff)) - Route Git scratch through configured volume ([block#1884](block#1884)) ([`2318b3096c`](block@2318b30)) - feat(relay): gate usage metrics behind stable leader ([block#1814](block#1814)) ([`59e9821503`](block@59e9821)) - Relay mesh: cross-pod tunnel + huddle transport (buzz-relay-mesh) ([block#1670](block#1670)) ([`ccb021d713`](block@ccb021d)) - feat(push): deliver accepted relay events as wakes ([block#1866](block#1866)) ([`bffbc5f22c`](block@bffbc5f)) - fix(db): resolve duplicate migration version ([block#1863](block#1863)) ([`08ad38a07f`](block@08ad38a)) - Add private product feedback sidecar ([block#1857](block#1857)) ([`af190c93e1`](block@af190c9)) - feat(relay): add durable community archival ([block#1834](block#1834)) ([`2b15a72675`](block@2b15a72)) - feat(push): add public APNs gateway ([block#1770](block#1770)) ([`1c006822e4`](block@1c00682)) - feat(relay): add atomic community ownership transfer ([block#1845](block#1845)) ([`52e42ccb9f`](block@52e42cc)) - Bound NIP-RS retention and search indexing ([block#1771](block#1771)) ([`1b4703021d`](block@1b47030)) - Add optional standalone pairing relay to Helm chart ([block#1799](block#1799)) ([`9b47c8548f`](block@9b47c85)) - fix(relay): publish membership snapshot on provisioning ([block#1761](block#1761)) ([`0950d392b7`](block@0950d39)) - feat(relay): per-community usage metrics ([block#1723](block#1723)) ([`620822899a`](block@6208228)) - refactor(desktop): remove vestigial MCP toolsets config ([block#1776](block#1776)) ([`dfec75b3c0`](block@dfec75b)) **To release:** merge this PR. The tag and build will happen automatically. Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
What
Emit per-community usage gauges and counter tags to Datadog so teams can answer "how many users / messages / active users does community X have right now" without manual SQL.
Metrics added
DB-polled gauges (new background poller,
BUZZ_USAGE_METRICS_INTERVAL_SECS, default 300 s, min 5 s)buzz_communities_totalCOUNT(*)on communitiesbuzz_community_userscommunity,type:human|agentdeactivated_at IS NULL; agent =agent_owner_pubkey IS NOT NULLbuzz_community_channelscommunity,type:stream|forum|dm|workflowdeleted_at IS NULLbuzz_community_messagescommunitykind=9,deleted_at IS NULLbuzz_community_relay_memberscommunity,rolebuzz_community_workflowscommunity,statusbuzz_community_git_reposcommunitybuzz_community_active_userscommunity,window:1d|7d|30d,type:human|agent|unknownCOUNT(DISTINCT pubkey)on events;unknown= pubkeys with nousersrow (profileless posters)buzz_community_active_channelscommunity,window:1d|7dFleet-wide totals (always emitted regardless of
BUZZ_USAGE_METRICS_PER_COMMUNITY)buzz_total_userstype:human|agentbuzz_total_channelstypebuzz_total_messagesbuzz_total_relay_membersrolebuzz_total_workflowsstatusbuzz_total_git_reposbuzz_total_active_userswindow,typebuzz_total_active_channelswindowbuzz_total_ws_connectionsbuzz_total_users_online_podbuzz_total_subscriptionsIn-memory snapshot gauges (same poller tick, from live state)
buzz_community_ws_connectionscommunitybuzz_community_users_online_podcommunitybuzz_community_subscriptionscommunitySnapshot pattern used instead of inc/dec to eliminate gauge drift from asymmetric call paths.
Write-path counter tags
communitytag added to:buzz_ws_connections_total(connection.rs)buzz_media_uploads_total(api/media.rs)buzz_workflow_runs_total(handlers/event.rs)New per-community event volume counter (kind-only counters remain fleet-wide to avoid kind × community cardinality explosion):
buzz_community_events_received_total{community}— per-community event throughputNew counters
buzz_users_created_total{community}— incremented only on first insert (ensure_userreturnstrue)buzz_channels_created_total{community, type}— incremented only on actual creation (ingest.rswas_createdpath, side_effects.rs new-channel paths)Bug fix
Kind
44200(agent turn metrics) now maps to"44200"inbounded_kind_labelinstead of collapsing to"other".Cardinality knob
BUZZ_USAGE_METRICS_PER_COMMUNITYcontrols how many per-community gauge series emit:all(default)offFleet-wide
buzz_total_*metrics always emit regardless of mode — Datadog cost lever without losing aggregate visibility.A
top:<k>mode (per-community series for the k most-active communities) is planned as a fast-follow once proper series-lifecycle semantics (gauge idle-timeout and stable pod-consistent tie-breaking) are designed.Implementation notes
ensure_user()return type changed fromResult<()>toResult<bool>(true = new insert, false =ON CONFLICT DO NOTHING). All callers updated.GROUP BY community_id— one query per metric family, not one query per community.metrics::gauge!(...).set(...)call. If any query fails the tick aborts cleanly — no mixed fresh/stale snapshot.rand::random::<u64>() % interval_secsgives each pod a unique start delay in[0, interval_secs)to prevent thundering herd on rolling deploy.MissedTickBehavior::Skipprevents catch-up burst.buzz_events_received_totalandbuzz_events_stored_totalkeep{kind}only (fleet-wide).bounded_kind_labelpasses through 10k values in20000..=29999(client-controlled); crossing withcommunitywould produce millions of series. Usebuzz_community_events_received_total{community}for per-community throughput.buzz_community_users_online_podis distinct within a pod (not fleet-distinct). Sum across pods for fleet total, with caveat that multi-pod pubkeys count N times.active_user_countsSQL uses a three-way LEFT JOIN classification:human(row exists, owner NULL),agent(row exists, owner NOT NULL),unknown(no row — profileless posters). Old query incorrectly counted no-row pubkeys as human.filter_maps emitwarn!log entries instead of being silently dropped.max:in Datadog (all pods report same DB value); in-memory gauges usesum:(per-pod partials). Documented in code comments.community(host string), nothost— avoids collision with Datadog's reserved infra host tag.