Skip to content

perf: optimize prebuilds membership reconciliation to check orgs not presets#20493

Merged
ssncferreira merged 3 commits into
mainfrom
ssncferreira/perf-prebuild-membership
Oct 29, 2025
Merged

perf: optimize prebuilds membership reconciliation to check orgs not presets#20493
ssncferreira merged 3 commits into
mainfrom
ssncferreira/perf-prebuild-membership

Conversation

@ssncferreira
Copy link
Copy Markdown
Contributor

@ssncferreira ssncferreira commented Oct 27, 2025

Description

The membership reconciliation ensures the prebuilds system user is a member of all organizations with prebuilds configured. To support prebuilds quota management, each organization must have a prebuilds group that the system user belongs to.

Problem

Previously, membership reconciliation iterated over all presets to check and update membership status. This meant database queries GetGroupByOrgAndName and InsertGroupMember were executed for each preset. Since presets are unique combinations of (organization, template, template version, preset), this resulted in several redundant checks for the same organization.

In dogfood, InsertGroupMember was called thousands of times per day, even though memberships were already configured (internal Grafana dashboard link)

Screenshot 2025-10-28 at 16 01 36

Solution

This PR introduces GetOrganizationsWithPrebuildStatus, a single query that returns:

  • All unique organizations with prebuilds configured
  • Whether the prebuilds user is a member of each organization
  • Whether the prebuilds group exists in each organization
  • Whether the prebuilds user is in the prebuilds group

The membership reconciliation logic now:

  • Fetches status for all organizations in one query
  • Only performs inserts for organizations missing required memberships or groups
  • Safely handles concurrent operations via unique constraint violations
  • This reduces database load from O(presets) to O(organizations) per reconciliation loop, with a single read query when everything is configured.

Changes

  • Add GetOrganizationsWithPrebuildStatus SQL query
  • Update membership.ReconcileAll to use organization-based reconciliation instead of preset-based
  • Update tests to reflect new behavior

Related to internal thread: https://codercom.slack.com/archives/C07GRNNRW03/p1760535570381369

@ssncferreira ssncferreira force-pushed the ssncferreira/perf-prebuild-membership branch from 1d3cbad to 9648569 Compare October 27, 2025 11:34
@ssncferreira ssncferreira force-pushed the ssncferreira/perf-prebuild-membership branch from 9648569 to 9d47f64 Compare October 28, 2025 15:40
@ssncferreira ssncferreira marked this pull request as ready for review October 28, 2025 16:12
Comment thread enterprise/coderd/prebuilds/membership.go
@ssncferreira ssncferreira merged commit 7e8fcb4 into main Oct 29, 2025
25 checks passed
@ssncferreira ssncferreira deleted the ssncferreira/perf-prebuild-membership branch October 29, 2025 14:24
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 29, 2025
@ssncferreira ssncferreira added the cherry-pick/v2.28 Needs to be cherry-picked to the 2.28 release branch label Oct 29, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cherry-pick/v2.28 Needs to be cherry-picked to the 2.28 release branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants