Skip to content

feat: record the connection method for session app usage - #31028

Draft
EhabY wants to merge 1 commit into
mainfrom
feat/session-app-method
Draft

EhabY wants to merge 1 commit into
mainfrom
feat/session-app-method

Conversation

@EhabY

@EhabY EhabY commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Store the connection method of each session app so insights no longer guess it from the app name at read time.

  • Coderd derives the method from the app name once, at ingest (codersdk.SessionAppMethod). The agent protocol, CLI, and API are unchanged: every current reporter's app name already implies its method.
  • template_usage_stats_session_apps gains method (CHECK on ssh, reconnecting_pty, port_forwarding) in its primary key. workspace_agent_stats.session_counts is nested as {method: {app: count}}.
  • The rollup and GetTemplateInsights group by (method, app). "Other apps" folds per method, and only a folded method's overflow is an upper bound.
  • Operational readers return a JSON array of the stored objects and coderd sums them per app (codersdk.SumSessionCounts), so metricscache, telemetry, and Prometheus output is unchanged.

Upgrade. Migration 000630 locks workspace_agent_stats in EXCLUSIVE mode (writes wait, reads continue) and converts only the rows the rollup and operational readers still read: from the last rollup minus 1 day, the window DeleteOldWorkspaceAgentStats keeps. It warns when the rollup is more than 24 hours behind, as 000590 did. A NOT VALID CHECK then rejects flat session counts, so a replica still on the previous release fails its stats writes instead of storing rows that would stall the new rollup. Until those replicas are replaced, their agent stats writes and session reads fail, as with 000590.

A future reporter (for example port-forward counting) only adds a name to the derivation rule; storage and insights already key by method. It will also need a method label in the UI, which keys rows by method/slug but does not show the method.

Closes #31012
Closes CLIENT-1052

Size

Files Lines
Production 11 +338 / -183
Tests (incl. dbgen) 12 +327 / -86
Generated 6 +143 / -138
Decisions and measurements

Decisions

  • D1: nest session_counts by method in place.
  • D2: cap over (method, app) pairs; one overflow row per method.
  • D3: derive the method at ingest. A protocol method field adds no information while every app name implies its method.
  • D4: no reader handles flat rows. Instead, convert the read window and reject flat writes with a CHECK, following the convert-then-constrain pattern of 000417 and the windowed conversion of 000590. Rows older than the window stay flat until purged; nothing reads them. The down migration drops the CHECK and skips them.
  • D5: operational readers return arrays of stored objects and coderd sums them. Expanding nested jsonb twice in SQL makes the planner estimate 100 x 100 rows per stat row, which pushes these queries past the JIT thresholds (the deployment reader went from 38 ms to 310 ms, all JIT compile time).

Migration (5M agent stat rows, 968k session app rows, local Postgres)

  • Converting workspace_agent_stats: about 43 s. Reads continued, inserts waited, and a flat insert after commit failed on workspace_agent_stats_session_counts_check.
  • template_usage_stats_session_apps: the column default fills existing rows without a rewrite, so only reconnecting_pty and port_forwarding rows are updated (about 1.3 s of blocked reads).
  • The CHECK is added last, in about 1 ms.

Queries, median ms (main / this PR)

5M rows, 2,000 agents, table clustered by created_at, EXPLAIN (ANALYZE) medians with the window pinned to the newest row:

Query main PR
UpsertTemplateUsageStats 2605 3063
GetTemplateInsights (7d) 390 383
GetTemplateInsightsByTemplate (5m, Prometheus; #29906 rewrites it) 30 34
GetDeploymentWorkspaceAgentStats 52 38
GetDeploymentWorkspaceAgentUsageStats 27 33
GetWorkspaceAgentUsageStats (1h) 567 253
GetWorkspaceAgentUsageStatsAndLabels (without the label joins) 8.6 5.8

GetWorkspaceAgentStats and GetWorkspaceAgentStatsAndLabels return jsonb_build_array of the latest row, at no measurable cost.

What keeps these close to main:

  • Nested expansion is avoided where it pushed plans past the JIT thresholds. The per-agent and deployment readers aggregate stored objects with jsonb_agg, and GetTemplateInsightsByTemplate expands app names in one call (jsonb_path_query_array(..., '$.*.keyvalue().key')).
  • The rollup's stale-row DELETE is bounded by latest_start. Without the bound, the overestimate made it scan the whole child table (573 ms down to 24 ms).
  • The overflow row in GetTemplateInsights uses grouped joins instead of per-method subqueries.

The rollup's remaining cost is the extra method level, which needs both keys.

Follow-ups

  • Port-forward counting needs a reporter and a UI method label (CLIENT-1053).

🤖 Generated by Coder Agents on behalf of @EhabY.

@linear-code

linear-code Bot commented Oct 19, 2026

Copy link
Copy Markdown

CLIENT-1052

Store the connection method of each session app in
template_usage_stats_session_apps and nest
workspace_agent_stats.session_counts by method. Coderd derives the
method from the app name at ingest, so the agent protocol, CLI, and API
are unchanged.

Migration 000630 blocks writes to workspace_agent_stats, converts the
rows the rollup and operational readers still read, and adds a NOT
VALID check that rejects flat session counts, so a coderd still on the
previous release cannot write rows the new readers fail on.

Insights group and fold "Other apps" per method. Operational readers
return the stored objects and coderd sums them per app, so
metricscache, telemetry, and Prometheus are unchanged.

Closes #31012
@EhabY
EhabY force-pushed the feat/session-app-method branch from 078f91c to df31332 Compare October 20, 2026 00:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Record the connection method for session app usage

1 participant