Skip to content
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .cursor/commands/pr-comments-review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Please review the PR comments with `gh pr status` and fix & resolve those issues that are valid and relevant. Leave those comments that are mostly bullshit unresolved. Report the result to me in detail. Do NOT automatically commit or stage the changes back to the PR!
6 changes: 5 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This file provides guidance to coding agents when working with code in this repo

#### Extra commands
These commands are usually already called by the user, but you can remind them to run it for you if they forgot to.
- **Build packages**: `pnpm build:packages`
- **Build packages**: `pnpm build:packages` (you should never call this yourself)
- **Start dependencies**: `pnpm restart-deps` (resets & restarts Docker containers for DB, Inbucket, etc. Usually already called by the user)
- **Run development**: Already called by the user in the background. You don't need to do this. This will also watch for changes and rebuild packages, codegen, etc. Do NOT call build:packages, dev, codegen, or anything like that yourself, as the dev is already running it.
- **Run minimal dev**: `pnpm dev:basic` (only backend and dashboard for resource-limited systems)
Expand Down Expand Up @@ -93,12 +93,16 @@ To see all development ports, refer to the index.html of `apps/dev-launchpad/pub
- If there is an external browser tool connected, use it to test changes you make to the frontend when possible.
- Whenever you update an SDK implementation in `sdks/implementations`, make sure to update the specs accordingly in `sdks/specs` such that if you reimplemented the entire SDK from the specs again, you would get the same implementation. (For example, if the specs are not precise enough to describe a change you made, make the specs more precise.)
- When building internal tools for Stack Auth developers (eg. internal interfaces like the WAL info log etc.): Make the interfaces look very concise, assume the user is a pro-user. This only applies to internal tools that are used primarily by Stack Auth developers.
- The dev server already builds the packages in the background whenever you update a file. If you run into issues with typechecking or linting in a dependency after updating something in a package, just wait a few seconds, and then try again, and they will likely be resolved.
- When asked to review PR comments, you can use `gh pr status` to get the current pull request you're working on.
- NEVER EVER AUTOMATICALLY COMMIT OR STAGE ANY CHANGES — DON'T MODIFY GIT WITHOUT USER CONSENT!
- When building frontend or React code for the dashboard, refer to DESIGN-GUIDE.md.
- NEVER implement a hacky solution without EXPLICIT approval from the user. Always go the extra mile to make sure the solution is clean, maintainable, and robust.
- Fail early, fail loud. Fail fast with an error instead of silently continuing.
- Do NOT use `as`/`any`/type casts or anything else like that to bypass the type system unless you specifically asked the user about it. Most of the time a place where you would use type casts is not one where you actually need them. Avoid wherever possible.
- When writing database migration files, assume that we have >1,000,000 rows in every table (unless otherwise specified). This means you may have to use CONDITIONALLY_REPEAT_MIGRATION_SENTINEL to avoid running the migration and things like concurrent index builds; see the existing migrations for examples.
- **When building frontend code, always carefully deal with loading and error states.** Be very explicit with these; some components make this easy, eg. the button onClick already takes an async callback for loading state, but make sure this is done everywhere, and make sure errors are NEVER just silently swallowed.
- Unless very clearly equivalent from types, prefer explicit null/undefinedness checks over boolean checks, eg. `foo == null` instead of `!foo`.

### Code-related
- Use ES6 maps instead of records wherever you can.
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
-- Migration to fix incorrectly formatted trusted domain entries in EnvironmentConfigOverride.
--
-- A previous migration sometimes generated entries like:
-- "domains.trustedDomains.<id>.<property1>": value1,
-- "domains.trustedDomains.<id>.<property2>": value2
--
-- Without the parent key:
-- "domains.trustedDomains.<id>": { ... }
--
-- This migration adds an empty object at the <id> level for any missing parent keys:
-- "domains.trustedDomains.<id>": {},
-- "domains.trustedDomains.<id>.<property1>": value1,
-- "domains.trustedDomains.<id>.<property2>": value2

-- Add temporary column to track processed rows (outside transaction so it's visible immediately)
-- SPLIT_STATEMENT_SENTINEL
-- SINGLE_STATEMENT_SENTINEL
-- RUN_OUTSIDE_TRANSACTION_SENTINEL
ALTER TABLE /* SCHEMA_NAME_SENTINEL */."EnvironmentConfigOverride" ADD COLUMN IF NOT EXISTS "temp_trusted_domains_checked" BOOLEAN DEFAULT FALSE;
-- SPLIT_STATEMENT_SENTINEL

-- Create index on the temporary column for efficient querying
-- SPLIT_STATEMENT_SENTINEL
-- SINGLE_STATEMENT_SENTINEL
-- RUN_OUTSIDE_TRANSACTION_SENTINEL
CREATE INDEX CONCURRENTLY IF NOT EXISTS "temp_eco_trusted_domains_checked_idx"
ON /* SCHEMA_NAME_SENTINEL */."EnvironmentConfigOverride" ("temp_trusted_domains_checked")
WHERE "temp_trusted_domains_checked" IS NOT TRUE;
-- SPLIT_STATEMENT_SENTINEL

-- Process rows in batches (outside transaction so each batch commits independently)
-- SPLIT_STATEMENT_SENTINEL
-- SINGLE_STATEMENT_SENTINEL
-- RUN_OUTSIDE_TRANSACTION_SENTINEL
-- CONDITIONALLY_REPEAT_MIGRATION_SENTINEL
WITH rows_to_check AS (
-- Get unchecked rows
SELECT "projectId", "branchId", "config"
FROM /* SCHEMA_NAME_SENTINEL */."EnvironmentConfigOverride"
WHERE "temp_trusted_domains_checked" IS NOT TRUE
-- Keep batch size small for consistent performance
LIMIT 1000
),
matching_keys AS (
-- Find all keys that look like "domains.trustedDomains.<id>.<property...>"
-- (4 or more dot-separated parts starting with domains.trustedDomains)
SELECT
rtc."projectId",
rtc."branchId",
key,
-- Extract the parent key: domains.trustedDomains.<id>
(string_to_array(key, '.'))[1] || '.' ||
(string_to_array(key, '.'))[2] || '.' ||
(string_to_array(key, '.'))[3] AS parent_key
FROM rows_to_check rtc,
jsonb_object_keys(rtc."config") AS key
WHERE key ~ '^domains\.trustedDomains\.[^.]+\..+'
-- Pattern matches: domains.trustedDomains.<id>.<anything>
-- e.g. "domains.trustedDomains.abc123.baseUrl"
),
missing_parents AS (
-- Find parent keys that don't exist in the config
SELECT DISTINCT
mk."projectId",
mk."branchId",
mk.parent_key
FROM matching_keys mk
JOIN rows_to_check rtc
ON rtc."projectId" = mk."projectId"
AND rtc."branchId" = mk."branchId"
WHERE NOT (rtc."config" ? mk.parent_key)
),
parents_to_add AS (
-- Aggregate all missing parent keys per row into a single jsonb object
SELECT
mp."projectId",
mp."branchId",
jsonb_object_agg(mp.parent_key, '{}'::jsonb) AS new_keys
FROM missing_parents mp
GROUP BY mp."projectId", mp."branchId"
),
updated_with_keys AS (
-- Update rows that need new parent keys
UPDATE /* SCHEMA_NAME_SENTINEL */."EnvironmentConfigOverride" eco
SET
"config" = eco."config" || pta.new_keys,
"updatedAt" = NOW(),
"temp_trusted_domains_checked" = TRUE
FROM parents_to_add pta
WHERE eco."projectId" = pta."projectId"
AND eco."branchId" = pta."branchId"
RETURNING eco."projectId", eco."branchId"
),
marked_as_checked AS (
-- Mark all checked rows (including ones that didn't need fixing)
UPDATE /* SCHEMA_NAME_SENTINEL */."EnvironmentConfigOverride" eco
SET "temp_trusted_domains_checked" = TRUE
FROM rows_to_check rtc
WHERE eco."projectId" = rtc."projectId"
AND eco."branchId" = rtc."branchId"
AND NOT EXISTS (
Comment thread
N2D4 marked this conversation as resolved.
SELECT 1 FROM updated_with_keys uwk
WHERE uwk."projectId" = eco."projectId"
AND uwk."branchId" = eco."branchId"
)
RETURNING eco."projectId"
)
SELECT COUNT(*) > 0 AS should_repeat_migration
FROM rows_to_check;
-- SPLIT_STATEMENT_SENTINEL

-- Clean up: drop temporary index (outside transaction since CREATE was also outside)
-- SPLIT_STATEMENT_SENTINEL
-- SINGLE_STATEMENT_SENTINEL
-- RUN_OUTSIDE_TRANSACTION_SENTINEL
DROP INDEX IF EXISTS "temp_eco_trusted_domains_checked_idx";
Comment thread
N2D4 marked this conversation as resolved.
Outdated
-- SPLIT_STATEMENT_SENTINEL

-- Clean up: drop temporary column (outside transaction)
-- SPLIT_STATEMENT_SENTINEL
-- SINGLE_STATEMENT_SENTINEL
-- RUN_OUTSIDE_TRANSACTION_SENTINEL
ALTER TABLE /* SCHEMA_NAME_SENTINEL */."EnvironmentConfigOverride" DROP COLUMN IF EXISTS "temp_trusted_domains_checked";
5 changes: 1 addition & 4 deletions apps/backend/scripts/db-migrations.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
import { applyMigrations } from "@/auto-migrations";
import { MIGRATION_FILES_DIR, getMigrationFiles } from "@/auto-migrations/utils";
import { Prisma } from "@/generated/prisma/client";
import { getClickhouseAdminClient } from "@/lib/clickhouse";
import { globalPrismaClient, globalPrismaSchema, sqlQuoteIdent } from "@/prisma-client";
import { spawnSync } from "child_process";
import fs from "fs";
import path from "path";
import * as readline from "readline";
import { seed } from "../prisma/seed";
import { getEnvVariable } from "@stackframe/stack-shared/dist/utils/env";
import { runClickhouseMigrations } from "./clickhouse-migrations";
import { getClickhouseAdminClient } from "@/lib/clickhouse";

const getClickhouseClient = () => getClickhouseAdminClient();

Expand Down Expand Up @@ -81,7 +80,6 @@ const generateMigrationFile = async () => {
const folderName = `${timestampPrefix()}_${migrationName}`;
const migrationDir = path.join(MIGRATION_FILES_DIR, folderName);
const migrationSqlPath = path.join(migrationDir, 'migration.sql');
const diffUrl = getEnvVariable('STACK_DATABASE_CONNECTION_STRING');

console.log(`Generating migration ${folderName}...`);
const diffResult = spawnSync(
Expand All @@ -92,7 +90,6 @@ const generateMigrationFile = async () => {
'migrate',
'diff',
'--from-config-datasource',
diffUrl,
'--to-schema',
'prisma/schema.prisma',
'--script',
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import { resetBranchConfigOverrideKeys, resetEnvironmentConfigOverrideKeys } from "@/lib/config";
import { createSmartRouteHandler } from "@/route-handlers/smart-route-handler";
import { adaptSchema, adminAuthTypeSchema, yupArray, yupNumber, yupObject, yupString } from "@stackframe/stack-shared/dist/schema-fields";

const levelSchema = yupString().oneOf(["branch", "environment"]).defined();

const levelConfigs = {
branch: {
reset: (options: { projectId: string, branchId: string, keysToReset: string[] }) =>
resetBranchConfigOverrideKeys(options),
},
environment: {
reset: (options: { projectId: string, branchId: string, keysToReset: string[] }) =>
resetEnvironmentConfigOverrideKeys(options),
},
};

export const POST = createSmartRouteHandler({
metadata: {
hidden: true,
summary: 'Reset config override keys',
description: 'Remove specific keys (and their nested descendants) from the config override at a given level. Uses the same nested key logic as the override algorithm.',
tags: ['Config'],
},
request: yupObject({
auth: yupObject({
type: adminAuthTypeSchema,
tenancy: adaptSchema,
}).defined(),
params: yupObject({
level: levelSchema,
}).defined(),
body: yupObject({
keys: yupArray(yupString().defined()).defined(),
}).defined(),
}),
response: yupObject({
statusCode: yupNumber().oneOf([200]).defined(),
bodyType: yupString().oneOf(["success"]).defined(),
}),
handler: async (req) => {
const levelConfig = levelConfigs[req.params.level];

await levelConfig.reset({
projectId: req.auth.tenancy.project.id,
branchId: req.auth.tenancy.branchId,
keysToReset: req.body.keys,
});

return {
statusCode: 200 as const,
bodyType: "success" as const,
};
},
});
2 changes: 1 addition & 1 deletion apps/backend/src/auto-migrations/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ export async function applyMigrations(options: {
}

for (const statementRaw of migration.sql.split('SPLIT_STATEMENT_SENTINEL')) {
const statement = statementRaw.replace('/* SCHEMA_NAME_SENTINEL */', sqlQuoteIdentToString(options.schema));
const statement = statementRaw.replaceAll('/* SCHEMA_NAME_SENTINEL */', sqlQuoteIdentToString(options.schema));
const runOutside = statement.includes('RUN_OUTSIDE_TRANSACTION_SENTINEL');
const isSingleStatement = statement.includes('SINGLE_STATEMENT_SENTINEL');
const isConditionallyRepeatMigration = statement.includes('CONDITIONALLY_REPEAT_MIGRATION_SENTINEL');
Expand Down
71 changes: 70 additions & 1 deletion apps/backend/src/lib/config.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { Prisma } from "@/generated/prisma/client";
import { Config, getInvalidConfigReason, normalize, override } from "@stackframe/stack-shared/dist/config/format";
import { Config, getInvalidConfigReason, normalize, override, removeKeysFromConfig } from "@stackframe/stack-shared/dist/config/format";
import { BranchConfigOverride, BranchConfigOverrideOverride, BranchIncompleteConfig, BranchRenderedConfig, CompleteConfig, EnvironmentConfigOverride, EnvironmentConfigOverrideOverride, EnvironmentIncompleteConfig, EnvironmentRenderedConfig, OrganizationConfigOverride, OrganizationConfigOverrideOverride, OrganizationIncompleteConfig, ProjectConfigOverride, ProjectConfigOverrideOverride, ProjectIncompleteConfig, ProjectRenderedConfig, applyBranchDefaults, applyEnvironmentDefaults, applyOrganizationDefaults, applyProjectDefaults, assertNoConfigOverrideErrors, branchConfigSchema, environmentConfigSchema, getConfigOverrideErrors, getIncompleteConfigWarnings, migrateConfigOverride, organizationConfigSchema, projectConfigSchema, sanitizeBranchConfig, sanitizeEnvironmentConfig, sanitizeOrganizationConfig, sanitizeProjectConfig } from "@stackframe/stack-shared/dist/config/schema";
import { ProjectsCrud } from "@stackframe/stack-shared/dist/interface/crud/projects";
import { branchConfigSourceSchema, yupBoolean, yupMixed, yupObject, yupRecord, yupString, yupUnion } from "@stackframe/stack-shared/dist/schema-fields";
Expand Down Expand Up @@ -457,6 +457,75 @@ export function overrideOrganizationConfigOverride(options: {
}


// ---------------------------------------------------------------------------------------------------------------------
// reset functions (remove specific keys from config override)
// ---------------------------------------------------------------------------------------------------------------------
// Uses the same nested key logic as the `override` function: resetting key "a.b" also resets "a.b.c".

export async function resetProjectConfigOverrideKeys(options: {
projectId: string,
keysToReset: string[],
}): Promise<void> {
// TODO put this in a serializable transaction (or a single SQL query) to prevent race conditions
const oldConfig = await rawQuery(globalPrismaClient, getProjectConfigOverrideQuery(options));
const newConfig = removeKeysFromConfig(oldConfig, options.keysToReset);

await setProjectConfigOverride({
projectId: options.projectId,
projectConfigOverride: newConfig as ProjectConfigOverride,
});
}

export async function resetBranchConfigOverrideKeys(options: {
projectId: string,
branchId: string,
keysToReset: string[],
}): Promise<void> {
// TODO put this in a serializable transaction (or a single SQL query) to prevent race conditions
const oldConfig = await rawQuery(globalPrismaClient, getBranchConfigOverrideQuery(options));
const newConfig = removeKeysFromConfig(oldConfig, options.keysToReset);

await setBranchConfigOverride({
projectId: options.projectId,
branchId: options.branchId,
branchConfigOverride: newConfig as BranchConfigOverride,
});
}

export async function resetEnvironmentConfigOverrideKeys(options: {
projectId: string,
branchId: string,
keysToReset: string[],
}): Promise<void> {
// TODO put this in a serializable transaction (or a single SQL query) to prevent race conditions
const oldConfig = await rawQuery(globalPrismaClient, getEnvironmentConfigOverrideQuery(options));
const newConfig = removeKeysFromConfig(oldConfig, options.keysToReset);

await setEnvironmentConfigOverride({
projectId: options.projectId,
branchId: options.branchId,
environmentConfigOverride: newConfig as EnvironmentConfigOverride,
});
}

export async function resetOrganizationConfigOverrideKeys(options: {
projectId: string,
branchId: string,
organizationId: string | null,
keysToReset: string[],
}): Promise<void> {
// TODO put this in a serializable transaction (or a single SQL query) to prevent race conditions
const oldConfig = await rawQuery(globalPrismaClient, getOrganizationConfigOverrideQuery(options));
const newConfig = removeKeysFromConfig(oldConfig, options.keysToReset);

await setOrganizationConfigOverride({
projectId: options.projectId,
branchId: options.branchId,
organizationId: options.organizationId,
organizationConfigOverride: newConfig as OrganizationConfigOverride,
});
}

// ---------------------------------------------------------------------------------------------------------------------
// internal functions
// ---------------------------------------------------------------------------------------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions apps/backend/src/lib/emails-low-level.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ async function _lowLevelSendEmailWithoutRetries(options: LowLevelSendEmailOption
}>> {
let finished = false;
runAsynchronously(async () => {
await wait(10000);
await wait(15_000);
if (!finished) {
captureError("email-send-timeout", new StackAssertionError("Email send took longer than 10s; maybe the email service is too slow?", {
captureError("email-send-timeout", new StackAssertionError("Email send took longer than 15s; maybe the email service is too slow?", {
config: options.emailConfig.type === 'shared' ? "shared" : pick(options.emailConfig, ['host', 'port', 'username', 'senderEmail', 'senderName']),
to: options.to,
subject: options.subject,
Expand Down
Loading
Loading