diff --git a/.agents/analytics.md b/.agents/analytics.md new file mode 100644 index 000000000..bcd12c824 --- /dev/null +++ b/.agents/analytics.md @@ -0,0 +1,375 @@ +# Analytics + +Reference for the TanStack.com event taxonomy in Google Analytics 4. + +**GA4 property:** `G-JMT1Z50SPS` +**First-party proxy:** all hits route through `/_a/g/collect` on tanstack.com (see `netlify.toml`) +**Implementation:** [src/utils/analytics.ts](../src/utils/analytics.ts), [src/utils/analytics/events.ts](../src/utils/analytics/events.ts) + +## Design principles + +1. **Event names describe what happened.** Properties carry context. Adding a new partner placement is an enum value, not a new event. +2. **One event per outcome, not per intent.** No `_clicked`/`_attempted` events when an outcome event is going to fire anyway. +3. **No per-row events.** Counts and joined-string arrays as properties, not N rows per generation. +4. **Session context propagates.** Slow-changing props like `mode_used` and `idea_used` are stamped on every builder event so any breakdown works without joins. +5. **Typed registry.** Wrong props for an event = TypeScript error, not silent bad data. + +--- + +## The funnel + +Application Builder, four steps. No `OR` conditions, no overlapping event names. + +| # | Step | Event | Filter | +| --- | --------------------- | ------------------- | --------------------------------- | +| 1 | Landed on builder | `page_view` | `page_type = application_builder` | +| 2 | Got an analysis | `builder_analyzed` | — | +| 3 | Got a generation | `builder_generated` | — | +| 4 | Took action on result | `builder_activated` | — | + +Drop-off between any two steps is unambiguous. + +**Failure rates** are separate explorations, not branches off the funnel: + +- Analysis failure rate = `builder_failed[stage=analysis]` ÷ (`builder_analyzed` + `builder_failed[stage=analysis]`) +- Generation failure rate = `builder_failed[stage=generation]` ÷ (`builder_generated` + `builder_failed[stage=generation]`) +- Login wall rate = `builder_failed[stage=login_blocked]` ÷ `page_view[page_type=application_builder]` + +--- + +## Event reference (9 events) + +Every event automatically receives `page_location`, `page_path`, `page_title`, `page_type` from the analytics utility. + +### `page_view` + +Fires on initial load (auto from gtag config) and on every SPA navigation. + +| Prop | Type | Notes | +| --------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------ | +| `page_location` | string | Full URL | +| `page_path` | string | Pathname | +| `page_title` | string | `document.title` | +| `page_type` | enum | `home`, `partners_index`, `partner_detail`, `blog_index`, `blog_post`, `docs`, `partners_embed`, `application_builder`, `page` | + +--- + +### `partner_viewed` + +A partner UI element scrolled ≥50% into the viewport. Fires once per element-mount per session (the underlying `IntersectionObserver` disconnects after first fire). + +| Prop | Type | Notes | +| ------------ | ------- | --------------------------------------------------- | +| `partner_id` | string | Stable partner identifier | +| `placement` | enum | See `PartnerPlacement` below | +| `slot_index` | number? | Position in the surface (0-indexed) when applicable | + +**Note on inflation:** when filters change on the partners directory, cards unmount/remount and `partner_viewed` re-fires. Treat session-unique impressions as the dedup'd metric (compute in BigQuery with `FIRST_VALUE(... PARTITION BY session_id, partner_id)`). + +--- + +### `partner_clicked` + +User clicked a partner UI element to navigate somewhere. + +| Prop | Type | Notes | +| ------------------ | ------- | -------------------------------------------------------------------------- | +| `partner_id` | string | | +| `placement` | enum | See `PartnerPlacement` below | +| `destination` | enum | `external` (partner's site) or `internal_detail` (our partner detail page) | +| `destination_host` | string? | Host of the destination URL when `external` | +| `slot_index` | number? | Position in the surface when applicable | + +CTR per placement = `partner_clicked` ÷ `partner_viewed` filtered to same `placement`. + +--- + +### `partner_filter_applied` + +User changed the filter state on the partners directory. + +| Prop | Type | Notes | +| ----------------- | -------------- | --------------------------------------------------------------------------------------------- | +| `change` | enum | `libraries_changed`, `status_changed`, `cleared_all` | +| `library_filters` | string | Comma-joined library ids in the filter, or empty string | +| `status_filter` | string \| null | `null` means "no status filter applied" — distinguishes from "user explicitly chose 'active'" | +| `result_count` | number | Number of partners visible after the filter | + +--- + +### `partner_inquiry_started` + +User clicked a "get in touch", "let's chat", or "become a partner" CTA. + +| Prop | Type | Notes | +| ----------- | ---- | ---------------------------------------------------------- | +| `placement` | enum | `partners_index_cta`, `library_callout`, `docs_right_rail` | + +--- + +### `builder_analyzed` + +Analysis API call succeeded. Outcome event — always preceded by user intent, no separate `_requested` event. + +| Prop | Type | Notes | +| ------------------------ | ------- | --------------------------------------------- | +| `mode_used` | enum | Session context: `lucky`, `confident`, `none` | +| `idea_used` | string | Session context: idea label, or `none` | +| `analysis_deployment` | string? | Inferred deploy target | +| `inferred_library_count` | number | | +| `inferred_partner_count` | number | | +| `feature_count` | number | | + +--- + +### `builder_generated` + +Generation API call succeeded. + +| Prop | Type | Notes | +| ----------------------- | ------- | ---------------------------------------------------------------------- | +| `mode_used` | enum | Session context | +| `idea_used` | string | Session context | +| `final_deployment` | string? | The chosen deploy target on the final result | +| `final_package_manager` | string | `pnpm`, `npm`, `yarn`, `bun` | +| `final_library_count` | number | | +| `final_partner_count` | number | | +| `final_addon_count` | number | | +| `library_ids` | string | Comma-joined LibraryIds — use `SPLIT()` in BigQuery for top-N analysis | +| `partner_ids` | string | Comma-joined | +| `addon_ids` | string | Comma-joined | + +--- + +### `builder_failed` + +Single umbrella event for analysis failures, generation failures, and login-wall blocks. Use the `stage` prop to distinguish. + +| Prop | Type | Notes | +| --------------------------------- | ------- | -------------------------------------------------------------------------------------------- | +| `mode_used` | enum | Session context | +| `idea_used` | string | Session context | +| `stage` | enum | `analysis`, `generation`, `login_blocked` | +| `error_message` | string? | Free-form error message — high cardinality, don't register as a dimension; query in BigQuery | +| `retry_after` | number? | Seconds until retry permitted (login_blocked only) | +| `anonymous_generations_remaining` | number? | When the failure was rate-limit related | + +--- + +### `builder_activated` + +User took an action on the generated result. Single event with `action` prop covers all post-generation actions. + +| Prop | Type | Notes | +| ----------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | +| `mode_used` | enum | Session context | +| `idea_used` | string | Session context | +| `action` | enum | See `BuilderAction` below | +| `surface` | enum | `result_panel` (main builder UI) or `deploy_dialog` | +| `provider` | string? | Deploy provider when applicable: `vercel`, `netlify`, `cloudflare` | +| `automatic` | boolean | `true` for system-driven actions (e.g., deploy_dialog auto-redirect countdown). Filter to `false` for true user click rates. | + +**Important:** automatic prompt-copies that fire as a side-effect of generation do NOT emit `builder_activated`. Only user-driven actions count as activation. + +--- + +## Enums + +### `PartnerPlacement` + +| Value | Where it appears | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| `directory` | Partner cards in `/partners` index | +| `detail` | Partner detail page CTA | +| `docs_rail` | Right rail on docs pages — partner cards AND the "Become a Partner" link both fire with this placement (event name distinguishes) | +| `blog_rail` | Right rail on blog pages | +| `grid` | Generic partners grid (fallback) | +| `home_grid` | Home page social-proof grid | +| `library_grid` | Library page partners section — filter `page_path` to know which library | +| `embed_grid` | Partners embed view | +| `docs_strip` | Mobile partner strip in docs | +| `ecosystem_game` | 3D ecosystem game islands | +| `partners_index_cta` | "Get in touch" mailto on `/partners` | +| `library_callout` | "Let's chat" callout per library | + +### `BuilderAction` + +| Value | Means | +| -------------------------- | ---------------------------------------------------------------------- | +| `copy_prompt` | User clicked a copy button on the prompt | +| `deploy` | Started a deploy through the deploy dialog | +| `clone_repo` | Cloned the GitHub repo | +| `open_codex` | Opened the result in Codex | +| `open_claude` | Opened the result in Claude | +| `open_cursor` | Opened the result in Cursor | +| `download` | Downloaded the project as a zip | +| `open_advanced` | Opened the advanced builder editor | +| `netlify_start` | Started a Netlify deploy from the result | +| `provider_redirect_manual` | User clicked through to deploy provider | +| `provider_redirect_auto` | Countdown auto-redirected user to deploy provider (`automatic = true`) | +| `open_repo` | Opened the project repo from the deploy dialog | + +--- + +## Session context + +`mode_used` and `idea_used` are tracked in the builder hook and stamped on **every** builder event. This means any builder event can be sliced by mode or idea without session joins. + +**`mode_used`** transitions: + +- `none` → `lucky` when user clicks "I'm feeling lucky" +- `none` → `confident` when user clicks "I'm feeling confident" + +**`idea_used`** transitions: + +- `none` → idea label string when user picks a suggested idea +- Reset back to `none` if user clears or types fresh input (TBD — currently sticks for the session) + +--- + +## Custom dimensions to register in GA4 + +Admin → Custom definitions → Create custom dimension. **Event scope** for all of these. Without registration, they're stored in BigQuery export but invisible in the GA4 UI. + +| Dimension name | API name | Used on | +| --------------------- | ----------------------- | -------------------------------------------------------- | +| Placement | `placement` | partner_viewed, partner_clicked, partner_inquiry_started | +| Partner ID | `partner_id` | partner_viewed, partner_clicked | +| Mode used | `mode_used` | all builder events | +| Idea used | `idea_used` | all builder events | +| Action | `action` | builder_activated | +| Surface | `surface` | builder_activated | +| Stage | `stage` | builder_failed | +| Final deployment | `final_deployment` | builder_generated | +| Final package manager | `final_package_manager` | builder_generated | +| Final library count | `final_library_count` | builder_generated | +| Final partner count | `final_partner_count` | builder_generated | +| Page type | `page_type` | all events | + +12 dimensions. Well under the 50-dimension event-scoped limit. + +**Don't register:** `error_message`, `library_ids`, `partner_ids`, `addon_ids`, `destination_host`. High cardinality. Query in BigQuery. + +--- + +## Common GA4 queries + +### "What's our funnel completion rate?" + +**Explore → Funnel exploration**, four steps as defined above. Open funnel. Show elapsed time. + +### "Lucky vs Confident: which mode converts better?" + +Same funnel, breakdown dropdown = `mode_used`. Compare side by side. + +### "Which deploy targets do successful generations land on?" + +**Explore → Free-form**, dimension = `final_deployment`, metric = event count of `builder_generated`. + +### "What % of users hit the login wall?" + +Free-form, filter `event_name = builder_failed`, breakdown by `stage`. + +### "Which placement converts best for partner discovery?" + +Free-form, filter `event_name = partner_clicked OR partner_viewed`, breakdown by `placement`. Compute CTR yourself: clicks ÷ views per placement. + +### "Top 10 libraries in completed generations" + +Requires BigQuery — `library_ids` is high-cardinality. Sample query: + +```sql +SELECT + library_id, + COUNT(*) AS generations +FROM `tanstack.analytics_*.events_*`, +UNNEST(SPLIT((SELECT value.string_value + FROM UNNEST(event_params) + WHERE key = 'library_ids'), ',')) AS library_id +WHERE event_name = 'builder_generated' + AND _TABLE_SUFFIX BETWEEN '20260101' AND '20260131' +GROUP BY library_id +ORDER BY generations DESC +LIMIT 10 +``` + +--- + +## BigQuery export + +**Strongly recommended.** Free at our event volume. Without it, anything beyond stock reports is painful. + +Setup: + +1. GA4 Admin → BigQuery Links → Link +2. Pick a GCP project, daily export, US multi-region +3. Tables appear at `tanstack.analytics_.events_YYYYMMDD` after ~24h + +Once enabled, all event properties are queryable — including the ones not registered as custom dimensions. Use SQL for everything dimensional or aggregate-heavy. Use the GA4 UI for funnel exploration and headline numbers. + +--- + +## Adding new events + +Anything additive should not require schema migration of the existing taxonomy. + +**Add a new partner placement:** + +1. Add the value to `PartnerPlacement` in [src/utils/analytics/events.ts](../src/utils/analytics/events.ts) +2. Use it at the call site +3. (Optional) update the placement table in this doc + +**Add a new builder action:** + +1. Add the value to `BuilderAction` +2. Use it at the `builder_activated` call site + +**Add a new event:** + +1. Add a new union member to `AnalyticsEvent` with its prop interface +2. Call `trackEvent({ name: '...', props: { ... } })` +3. Register relevant breakdown props as custom dimensions in GA4 admin +4. Document the event in this file + +**Don't:** add new properties to existing events without updating both the type definition and this doc. Schema drift in analytics events is the slowest bug to detect. + +--- + +## Code locations + +| File | Purpose | +| ------------------------------------------------------------------------------------- | -------------------------------------------------------------------- | +| [src/utils/analytics.ts](../src/utils/analytics.ts) | `trackEvent`, `useTrackedImpression`, `trackPageView`, `getPageType` | +| [src/utils/analytics/events.ts](../src/utils/analytics/events.ts) | Typed event registry — discriminated union of all events | +| [src/utils/analytics/providers/google.ts](../src/utils/analytics/providers/google.ts) | gtag wrapper | +| [src/utils/analytics/types.ts](../src/utils/analytics/types.ts) | Provider interface | +| [src/routes/\_\_root.tsx](../src/routes/__root.tsx) | gtag bootstrap, `PageViewTracker` | +| [netlify.toml](../netlify.toml) | First-party proxy redirects | + +--- + +## Migration history + +### 2026-05 — schema v2 + +Collapsed 28 events into 9. Removed `application_starter_*` event family. Replaced with `builder_*` events. Mode and idea selection moved from standalone events into session-context props on every builder event. + +**Events removed entirely** (no replacement): + +- `application_starter_library_toggled`, `_integration_toggled`, `_package_manager_toggled`, `_toolchain_toggled` — config exploration depth no longer tracked. Final config is on `builder_generated`. +- `application_starter_continue_clicked`, `_generate_clicked` — intent implied by outcome events. +- `application_starter_login_clicked`, `_value_copied` (auto), `_builder_result_applied`, `_final_partner_in_prompt`, `_final_addon_in_prompt`. + +**Events folded into others:** + +- `application_starter_action_clicked` (mode_selected) → `mode_used` prop on subsequent events +- `application_starter_idea_selected` → `idea_used` prop on subsequent events +- `application_starter_login_required` → `builder_failed[stage=login_blocked]` +- `application_starter_value_copied` (user trigger) → `builder_activated[action=copy_prompt]` +- All other `application_starter_action_clicked` calls → `builder_activated` +- `partner_card_clicked`, `partner_click` → `partner_clicked` +- `partner_impression`, `partner_detail_viewed` → `partner_viewed` +- `partners_filter_changed` → `partner_filter_applied` +- `partner_inquiry_clicked`, `become_partner_clicked` → `partner_inquiry_started` + +Historical data with old event names is still queryable in GA4 and BigQuery. Cutover date: see git log for the migration commit. diff --git a/.agents/index.md b/.agents/index.md new file mode 100644 index 000000000..01316aeb9 --- /dev/null +++ b/.agents/index.md @@ -0,0 +1,19 @@ +# Agent Guidelines + +TanStack.com marketing site built with TanStack Start. + +## Essentials + +- Package manager: `pnpm` +- Run `pnpm test` before commits not after every tiny edit +- Don't run builds or tests after every change. This is a visual site; assume changes work unless reported otherwise. +- Rely on `tsc` or your built-in LSP integration (hopefully you have this) +- **Typesafety is paramount.** Never cast types; fix at source instead. See [typescript.md](./typescript.md). + +## Topic Guides + +- [TypeScript Conventions](./typescript.md): Type inference, casting rules, generic naming +- [TanStack Patterns](./tanstack-patterns.md): Loaders, server functions, environment shaking +- [UI Style Guide](./ui-style.md): Visual design principles for 2026 +- [Workflow](./workflow.md): Build commands, debugging, Playwright +- [Analytics](./analytics.md): GA4 event taxonomy, funnel definition, custom dimensions diff --git a/.agents/tanstack-patterns.md b/.agents/tanstack-patterns.md new file mode 100644 index 000000000..d0ff4e278 --- /dev/null +++ b/.agents/tanstack-patterns.md @@ -0,0 +1,76 @@ +# TanStack Patterns + +## loaderDeps Must Be Specific + +Only include properties actually used in the loader. This ensures proper cache invalidation. + +```typescript +// Bad: includes everything +loaderDeps: ({ search }) => search, +loader: async ({ deps }) => { + await fetchData({ page: deps.page, pageSize: deps.pageSize }) +} + +// Good: only what's used +loaderDeps: ({ search }) => ({ + page: search.page, + pageSize: search.pageSize, +}), +loader: async ({ deps }) => { + await fetchData({ page: deps.page, pageSize: deps.pageSize }) +} +``` + +## Loaders Are Isomorphic + +Loaders run on both server and client. They cannot directly access server-only APIs. + +```typescript +// Bad: direct server API access +loader: async () => { + const data = await fs.readFile('data.json') + return { data } +} + +// Good: call a server function +loader: async () => { + const data = await serverFn({ data: { id: '123' } }) + return { data } +} +``` + +## Environment Shaking + +TanStack Start strips any code not referenced by a `createServerFn` handler from the client build. + +- Server-only code (database, fs) is automatically excluded from client bundles +- Only code inside `createServerFn` handlers goes to server bundles +- Code outside handlers is included in both bundles + +## Importing Server Functions + +Server functions wrapped in `createServerFn` can be imported statically. Never use dynamic imports for server-only code in components. + +```typescript +// Bad: dynamic import causes bundler issues +const rolesQuery = useQuery({ + queryFn: async () => { + const { listRoles } = await import('~/utils/roles.server') + return listRoles({ data: {} }) + }, +}) + +// Good: static import +import { listRoles } from '~/utils/roles.server' + +const rolesQuery = useQuery({ + queryFn: async () => listRoles({ data: {} }), +}) +``` + +## Server-Only Import Rules + +1. `createServerFn` wrappers can be imported statically anywhere +2. Direct server-only code (database clients, fs) must only be imported: + - Inside `createServerFn` handlers + - In `*.server.ts` files diff --git a/.agents/typescript.md b/.agents/typescript.md new file mode 100644 index 000000000..e63301ec4 --- /dev/null +++ b/.agents/typescript.md @@ -0,0 +1,45 @@ +# TypeScript Conventions + +## Avoid Type Casting + +Never cast types unless absolutely necessary. This includes: + +- Manual generic type parameters (e.g., ``) +- Type assertions using `as` +- Type assertions using `satisfies` + +## Prefer Type Inference + +Infer types by going up the logical chain: + +1. **Schema validation** as source of truth (Convex, Zod, etc.) +2. **Type inference** from function return types, API responses +3. **Fix at source** (schema, API definition, function signature) rather than casting at point of use + +```typescript +// Bad +const result = api.getData() as MyType +const value = getValue() + +// Good +const result = api.getData() // Type inferred from return type +const value = getValue() // Type inferred from implementation +``` + +## Generic Type Parameter Naming + +All generic type parameters must be prefixed with `T`. + +```typescript +// Bad +function withCapability( + handler: (user: AuthUser, ...args: Args) => R, +) { ... } + +// Good +function withCapability( + handler: (user: AuthUser, ...args: TArgs) => TReturn, +) { ... } +``` + +Common names: `T`, `TArgs`, `TReturn`, `TData`, `TError`, `TKey`, `TValue` diff --git a/.agents/ui-style.md b/.agents/ui-style.md new file mode 100644 index 000000000..d696b3346 --- /dev/null +++ b/.agents/ui-style.md @@ -0,0 +1,52 @@ +# UI Style Guide 2026 + +## Layout + +- Fewer, well-defined containers over many small sections +- Generous spacing creates separation before adding visual effects +- Cards are acceptable when they express grouping or hierarchy + +## Corners + +- Rounded corners are standard +- Subtle radius values that feel intentional, not playful +- Avoid sharp 90-degree corners unless intentionally industrial + +## Shadows and Depth + +- Soft, low-contrast, diffused shadows +- Shadows imply separation, not elevation theatrics +- No heavy drop shadows or strong directional lighting +- One to two shadow layers max + +## Cards + +- Cards should feel grounded, not floating +- Light elevation, border plus shadow, or surface contrast +- Don't overuse cards as a default layout primitive + +## Color and Surfaces + +- Soft neutrals, off-whites, warm grays +- Surface contrast or translucency instead of strong outlines +- Glass/frosted effects acceptable when subtle and accessible + +## Interaction + +- Micro transitions reinforce spatial relationships +- Hover/focus states feel responsive, not animated +- No excessive motion or springy effects + +## Typography + +- Strong headings, calm body text +- No visual noise around content + +## What to Avoid + +- Chunky shadows +- Overly flat, sterile layouts +- Neumorphism as primary style +- Over-designed card grids + +**Summary: If depth does not improve comprehension, remove it.** diff --git a/.agents/workflow.md b/.agents/workflow.md new file mode 100644 index 000000000..07838ff50 --- /dev/null +++ b/.agents/workflow.md @@ -0,0 +1,46 @@ +# Workflow + +## Build Commands + +- `pnpm test`: Run at end of task batches +- `pnpm build`: Only for build/bundler issues or verifying production output +- `pnpm lint`: Check for code issues +- `dev` runs indefinitely in watch mode + +Don't build after every change. This is a visual site; assume changes work. + +## Dev Authentication + +The dev server uses the real production database and OAuth. There are no auth bypasses. + +To authenticate a dev session, the human must run: + +```sh +pnpm auth:login +``` + +This opens `tanstack.com`, completes OAuth, and saves `DEV_SESSION_TOKEN` to `.env.local`. The dev server then auto-injects that token as a session cookie on startup. + +If the user asks for help with anything requiring authentication (account pages, admin, mutations) and `DEV_SESSION_TOKEN` is not set in `.env.local`, tell them to run `pnpm auth:login` first and restart the dev server. + +## Debugging Visual Issues + +When something doesn't work or look right: + +1. Use Playwright MCP to view the page and debug visually +2. Use `pnpm build` only for build/bundler issues +3. Use `pnpm lint` for code issues + +## Playwright + +Playwright is available via `npx playwright` and should be used freely to interact with the running dev server like a human would. This includes testing, debugging, visual verification, design iteration, and general development workflows. + +Since the dev server runs with a real authenticated session (via `DEV_SESSION_TOKEN`), Playwright has full access to gated pages, account features, admin areas, and authenticated mutations — exactly as the signed-in user would. + +Common uses: + +- `npx playwright screenshot ` — capture current state of any page +- Navigate to a page, interact with elements, re-screenshot to verify changes +- Debug layout/visual issues without needing a human to look +- Verify authenticated flows end-to-end (account, showcase submissions, admin, etc.) +- Iterate on UI by screenshotting, editing, screenshotting again diff --git a/.claude/launch.json b/.claude/launch.json new file mode 100644 index 000000000..7e34cf155 --- /dev/null +++ b/.claude/launch.json @@ -0,0 +1,17 @@ +{ + "version": "0.0.1", + "configurations": [ + { + "name": "web", + "runtimeExecutable": "pnpm", + "runtimeArgs": ["dev"], + "port": 3000 + }, + { + "name": "db-studio", + "runtimeExecutable": "pnpm", + "runtimeArgs": ["db:studio"], + "port": 4983 + } + ] +} diff --git a/.claude/tanstack-patterns.md b/.claude/tanstack-patterns.md new file mode 120000 index 000000000..7335c6c22 --- /dev/null +++ b/.claude/tanstack-patterns.md @@ -0,0 +1 @@ +../.agents/tanstack-patterns.md \ No newline at end of file diff --git a/.claude/typescript.md b/.claude/typescript.md new file mode 120000 index 000000000..bf4e25139 --- /dev/null +++ b/.claude/typescript.md @@ -0,0 +1 @@ +../.agents/typescript.md \ No newline at end of file diff --git a/.claude/ui-style.md b/.claude/ui-style.md new file mode 120000 index 000000000..f59b7412f --- /dev/null +++ b/.claude/ui-style.md @@ -0,0 +1 @@ +../.agents/ui-style.md \ No newline at end of file diff --git a/.claude/workflow.md b/.claude/workflow.md new file mode 120000 index 000000000..c19e63b75 --- /dev/null +++ b/.claude/workflow.md @@ -0,0 +1 @@ +../.agents/workflow.md \ No newline at end of file diff --git a/.github/workflows/autofix.yml b/.github/workflows/autofix.yml index e4a484469..b336e672a 100644 --- a/.github/workflows/autofix.yml +++ b/.github/workflows/autofix.yml @@ -18,11 +18,12 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v5.0.0 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: fetch-depth: 0 + persist-credentials: false - name: Setup Tools - uses: tanstack/config/.github/setup@main + uses: tanstack/config/.github/setup@e4b48f16568324f76f467aa4c2aac2f05db632c3 # main - name: Fix formatting run: pnpm format - name: Apply fixes diff --git a/.github/workflows/docs-links.yml b/.github/workflows/docs-links.yml new file mode 100644 index 000000000..4771fdd25 --- /dev/null +++ b/.github/workflows/docs-links.yml @@ -0,0 +1,42 @@ +name: Docs Links + +on: + workflow_dispatch: + schedule: + - cron: '0 9 * * *' + +permissions: + contents: read + +jobs: + docs-links: + name: Docs Links + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + - name: Setup Tools + uses: tanstack/config/.github/setup@e4b48f16568324f76f467aa4c2aac2f05db632c3 # main + + - name: Check docs menu links + env: + GITHUB_TOKEN: ${{ github.token }} + run: | + set +e + pnpm docs:links:check -- --json docs-menu-link-report.json + status=$? + + if [ -f docs-menu-link-report.json ]; then + { + echo "### Docs menu link report" + echo + echo '```json' + cat docs-menu-link-report.json + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + fi + + exit $status diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 41aec48b4..d4497f807 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -3,17 +3,21 @@ name: PR on: pull_request: +permissions: + contents: read + jobs: pr: name: PR runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v5.0.0 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: fetch-depth: 0 + persist-credentials: false - name: Setup Tools - uses: tanstack/config/.github/setup@main + uses: tanstack/config/.github/setup@e4b48f16568324f76f467aa4c2aac2f05db632c3 # main - name: Run Build run: pnpm build - name: Run Tests diff --git a/.github/workflows/update-tanstack-deps.yml b/.github/workflows/update-tanstack-deps.yml index 60f23f758..7d4ad61d9 100644 --- a/.github/workflows/update-tanstack-deps.yml +++ b/.github/workflows/update-tanstack-deps.yml @@ -10,17 +10,20 @@ jobs: update-deps: name: Update TanStack Dependencies runs-on: ubuntu-latest + permissions: + contents: write steps: - name: Git Checkout - uses: actions/checkout@v4 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: - token: ${{ secrets.GITHUB_TOKEN }} + # This scheduled job commits dependency updates back to the branch. + persist-credentials: true - name: Setup pnpm - uses: pnpm/action-setup@v4 + uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4.4.0 - name: Setup Node - uses: actions/setup-node@v4 + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version-file: .nvmrc cache: pnpm diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml new file mode 100644 index 000000000..1d4088db8 --- /dev/null +++ b/.github/workflows/zizmor.yml @@ -0,0 +1,25 @@ +name: GitHub Actions Security Analysis + +on: + push: + branches: [main] + pull_request: + branches: ['**'] + +permissions: {} + +jobs: + zizmor: + name: Run zizmor + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + - name: Run zizmor + uses: zizmorcore/zizmor-action@b1d7e1fb5de872772f31590499237e7cce841e8e # v0.5.3 + with: + advanced-security: false + annotations: true diff --git a/.gitignore b/.gitignore index f7793cf5a..7021fe7e7 100644 --- a/.gitignore +++ b/.gitignore @@ -7,12 +7,14 @@ drizzle/migrations .DS_Store .cache .env +.env.* .vercel .output .vinxi .tanstack-start/build .nitro/* .netlify/* +.wrangler/* /build/ /api/ @@ -30,5 +32,8 @@ dist test-results .claude/CLAUDE.md +.claude/worktrees .eslintcache .tsbuildinfo +src/routeTree.gen.ts +.og-preview/ diff --git a/.nvmrc b/.nvmrc index ae0e6a25c..d9b042470 100644 --- a/.nvmrc +++ b/.nvmrc @@ -1 +1 @@ -v20.19.6 +v25.9.0 diff --git a/.oxfmtrc.json b/.oxfmtrc.json new file mode 100644 index 000000000..ac154e249 --- /dev/null +++ b/.oxfmtrc.json @@ -0,0 +1,20 @@ +{ + "$schema": "./node_modules/oxfmt/configuration_schema.json", + "semi": false, + "singleQuote": true, + "trailingComma": "all", + "printWidth": 80, + "sortPackageJson": false, + "ignorePatterns": [ + "**/api", + "**/build", + "**/public", + "pnpm-lock.yaml", + "routeTree.gen.ts", + "src/blog/tanstack-db-0.1-the-embedded-client-database-for-tanstack-query.md", + ".content-collections", + ".claude", + "dist/**", + ".output/**" + ] +} diff --git a/.oxlintrc.json b/.oxlintrc.json new file mode 100644 index 000000000..f6467d88c --- /dev/null +++ b/.oxlintrc.json @@ -0,0 +1,204 @@ +{ + "$schema": "./node_modules/oxlint/configuration_schema.json", + "plugins": ["typescript"], + "categories": { + "correctness": "off" + }, + "options": { + "typeAware": true, + "typeCheck": true + }, + "env": { + "builtin": true + }, + "ignorePatterns": [ + "node_modules", + "dist", + "build", + ".content-collections", + ".tanstack-start", + ".netlify", + "public", + "convex/.temp", + ".claude" + ], + "rules": { + "no-array-constructor": "error", + "no-unused-expressions": "error", + "no-unused-vars": "error", + "typescript/ban-ts-comment": "error", + "typescript/no-duplicate-enum-values": "error", + "typescript/no-empty-object-type": "error", + "typescript/no-explicit-any": "error", + "typescript/no-extra-non-null-assertion": "error", + "typescript/no-misused-new": "error", + "typescript/no-namespace": "error", + "typescript/no-non-null-asserted-optional-chain": "error", + "typescript/no-require-imports": "error", + "typescript/no-this-alias": "error", + "typescript/no-unnecessary-type-constraint": "error", + "typescript/no-unsafe-declaration-merging": "error", + "typescript/no-unsafe-function-type": "error", + "typescript/no-wrapper-object-types": "error", + "typescript/prefer-as-const": "error", + "typescript/prefer-namespace-keyword": "error", + "typescript/triple-slash-reference": "error" + }, + "overrides": [ + { + "files": ["**/*.{js,jsx}"], + "rules": { + "constructor-super": "error", + "for-direction": "error", + "no-async-promise-executor": "error", + "no-case-declarations": "error", + "no-class-assign": "error", + "no-compare-neg-zero": "error", + "no-cond-assign": "error", + "no-const-assign": "error", + "no-constant-binary-expression": "error", + "no-constant-condition": "error", + "no-control-regex": "error", + "no-debugger": "error", + "no-delete-var": "error", + "no-dupe-class-members": "error", + "no-dupe-else-if": "error", + "no-dupe-keys": "error", + "no-duplicate-case": "error", + "no-empty": "error", + "no-empty-character-class": "error", + "no-empty-pattern": "error", + "no-empty-static-block": "error", + "no-ex-assign": "error", + "no-extra-boolean-cast": "error", + "no-fallthrough": "error", + "no-func-assign": "error", + "no-global-assign": "error", + "no-import-assign": "error", + "no-invalid-regexp": "error", + "no-irregular-whitespace": "error", + "no-loss-of-precision": "error", + "no-misleading-character-class": "error", + "no-new-native-nonconstructor": "error", + "no-nonoctal-decimal-escape": "error", + "no-obj-calls": "error", + "no-prototype-builtins": "error", + "no-redeclare": "error", + "no-regex-spaces": "error", + "no-self-assign": "error", + "no-setter-return": "error", + "no-shadow-restricted-names": "error", + "no-sparse-arrays": "error", + "no-this-before-super": "error", + "no-unexpected-multiline": "error", + "no-unsafe-finally": "error", + "no-unsafe-negation": "error", + "no-unsafe-optional-chaining": "error", + "no-unused-labels": "error", + "no-unused-private-class-members": "error", + "no-useless-backreference": "error", + "no-useless-catch": "error", + "no-useless-escape": "error", + "no-with": "error", + "require-yield": "error", + "use-isnan": "error", + "valid-typeof": "error" + } + }, + { + "files": ["**/*.ts", "**/*.tsx", "**/*.mts", "**/*.cts"], + "rules": { + "constructor-super": "off", + "no-class-assign": "off", + "no-const-assign": "off", + "no-dupe-class-members": "off", + "no-dupe-keys": "off", + "no-func-assign": "off", + "no-import-assign": "off", + "no-new-native-nonconstructor": "off", + "no-obj-calls": "off", + "no-redeclare": "off", + "no-setter-return": "off", + "no-this-before-super": "off", + "no-unsafe-negation": "off", + "no-var": "error", + "no-with": "off", + "prefer-const": "error", + "prefer-rest-params": "error", + "prefer-spread": "error" + } + }, + { + "files": ["**/*.{ts,tsx}"], + "rules": { + "no-redeclare": "error", + "no-shadow": "off", + "no-unused-vars": [ + "warn", + { + "argsIgnorePattern": "(^_)|(^__+$)|(^e$)|(^error$)", + "varsIgnorePattern": "(^_)|(^__+$)|(^e$)|(^error$)", + "caughtErrorsIgnorePattern": "(^_)|(^__+$)|(^e$)|(^error$)" + } + ], + "typescript/no-explicit-any": "off" + } + }, + { + "files": ["**/*.{ts,tsx,js,jsx}"], + "rules": { + "jsx-a11y/alt-text": "error", + "jsx-a11y/anchor-ambiguous-text": "off", + "jsx-a11y/anchor-has-content": "error", + "jsx-a11y/anchor-is-valid": "error", + "jsx-a11y/aria-activedescendant-has-tabindex": "error", + "jsx-a11y/aria-props": "error", + "jsx-a11y/aria-proptypes": "error", + "jsx-a11y/aria-role": "error", + "jsx-a11y/aria-unsupported-elements": "error", + "jsx-a11y/autocomplete-valid": "error", + "jsx-a11y/click-events-have-key-events": "error", + "jsx-a11y/heading-has-content": "error", + "jsx-a11y/html-has-lang": "error", + "jsx-a11y/iframe-has-title": "error", + "jsx-a11y/img-redundant-alt": "error", + "jsx-a11y/label-has-associated-control": "error", + "jsx-a11y/media-has-caption": "error", + "jsx-a11y/mouse-events-have-key-events": "error", + "jsx-a11y/no-access-key": "error", + "jsx-a11y/no-autofocus": "error", + "jsx-a11y/no-distracting-elements": "error", + "jsx-a11y/no-noninteractive-tabindex": [ + "error", + { + "tags": [], + "roles": ["tabpanel"], + "allowExpressionValues": true + } + ], + "jsx-a11y/no-redundant-roles": "error", + "jsx-a11y/no-static-element-interactions": [ + "error", + { + "allowExpressionValues": true, + "handlers": [ + "onClick", + "onMouseDown", + "onMouseUp", + "onKeyPress", + "onKeyDown", + "onKeyUp" + ] + } + ], + "jsx-a11y/role-has-required-aria-props": "error", + "jsx-a11y/role-supports-aria-props": "error", + "jsx-a11y/scope": "error", + "jsx-a11y/tabindex-no-positive": "error", + "react/rules-of-hooks": "error", + "react/exhaustive-deps": "warn" + }, + "plugins": ["react", "jsx-a11y"] + } + ] +} diff --git a/.prettierignore b/.prettierignore deleted file mode 100644 index 48a895e6e..000000000 --- a/.prettierignore +++ /dev/null @@ -1,10 +0,0 @@ -**/api -**/build -**/public -pnpm-lock.yaml -routeTree.gen.ts -src/blog/tanstack-db-0.1-the-embedded-client-database-for-tanstack-query.md -.content-collections -.claude -dist/** -.output/** \ No newline at end of file diff --git a/.prettierrc b/.prettierrc deleted file mode 100644 index e3b414c7e..000000000 --- a/.prettierrc +++ /dev/null @@ -1,5 +0,0 @@ -{ - "semi": false, - "singleQuote": true, - "trailingComma": "all" -} diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index 71f08aceb..000000000 --- a/AGENTS.md +++ /dev/null @@ -1,330 +0,0 @@ -# Agent Guidelines - -## Typesafety - -**Typesafety is of utmost importance.** - -### Avoid Type Casting - -We **never ever** cast types unless it's absolutely necessary. This includes: - -- Manual generic type parameters (e.g., ``) -- Type assertions using `as` -- Type assertions using `satisfies` -- Any other form of type casting - -### Prefer Type Inference - -Always infer types and go up the logical chain as far as we can control to determine types. The preferred approach is: - -1. **Schema validation** - Use schema definitions (e.g., Convex schema, Zod, etc.) as the source of truth -2. **Type inference from concrete sources** - Let TypeScript infer types from function return types, API responses, etc. -3. **Go up the chain** - Trace types back to their source rather than casting at the point of use - -### Example - -❌ **Bad:** - -```typescript -const result = api.getData() as MyType -const value = getValue() -``` - -✅ **Good:** - -```typescript -// Infer from schema or API definition -const result = api.getData() // Type inferred from api.getData return type -const value = getValue() // Type inferred from function implementation -``` - -If types need to be fixed, fix them at the source (schema, API definition, function signature) rather than casting at the point of use. - -### Generic Type Parameter Naming - -**All generic type parameters must be prefixed with `T`.** - -This convention makes it immediately clear that a name refers to a type parameter rather than a concrete type or value. - -❌ **Bad:** - -```typescript -function withCapability( - handler: (user: AuthUser, ...args: Args) => R, -) { ... } -``` - -✅ **Good:** - -```typescript -function withCapability( - handler: (user: AuthUser, ...args: TArgs) => TReturn, -) { ... } -``` - -Common examples: - -- `T` for a single generic type -- `TArgs` for argument types -- `TReturn` for return types -- `TData` for data types -- `TError` for error types -- `TKey` for key types -- `TValue` for value types - -## Route Loaders - -### loaderDeps Must Be Specific - -**loaderDeps must always be specific to what's actually used in the loader.** - -Only include the properties from `search` (or other sources) that are actually used in the loader function. This ensures proper cache invalidation and prevents unnecessary re-runs when unrelated search params change. - -❌ **Bad:** - -```typescript -loaderDeps: ({ search }) => search, // Includes everything, even unused params -loader: async ({ deps }) => { - // Only uses deps.page and deps.pageSize - await fetchData({ page: deps.page, pageSize: deps.pageSize }) -} -``` - -✅ **Good:** - -```typescript -loaderDeps: ({ search }) => ({ - page: search.page, - pageSize: search.pageSize, - // Only include what's actually used in the loader -}), -loader: async ({ deps }) => { - await fetchData({ page: deps.page, pageSize: deps.pageSize }) -} -``` - -This ensures the loader only re-runs when the specific dependencies change, not when unrelated search params (like `expanded`, `viewMode`, etc.) change. - -### Loaders Are Isomorphic - -**Loaders in TanStack Start/Router are isomorphic and cannot call server logic unless via a call to a server function.** - -Loaders run on both the server and client, so they cannot directly access server-only APIs (like file system, database connections, etc.). To perform server-side operations, loaders must call server functions (e.g., TanStack server functions created via `createServerFn()`, API routes, or other server functions). - -❌ **Bad:** - -```typescript -loader: async () => { - // This won't work - direct server API access - const data = await fs.readFile('data.json') - return { data } -} -``` - -✅ **Good:** - -```typescript -loader: async () => { - // Call a server function instead - // TanStack server functions created via createServerFn() can be called directly - const data = await serverFn({ data: { id: '123' } }) - return { data } -} -``` - -## Server-Only Code and Environment Shaking - -### TanStack Start Environment Shaking - -**TanStack Start performs environment shaking - any code not referenced by a `createServerFn` handler is stripped from the client build.** - -This means: - -- Server-only code (database, file system, etc.) is automatically excluded from client bundles -- Only code inside `createServerFn` handlers is included in server bundles -- Code outside handlers is included in both server and client bundles - -### Importing Server Functions - -**Server functions wrapped in `createServerFn` can be safely imported statically in route files.** - -❌ **Bad - Dynamic imports in component code:** - -```typescript -// In a route component -const rolesQuery = useQuery({ - queryFn: async () => { - const { listRoles } = await import('~/utils/roles.server') - return listRoles({ data: {} }) - }, -}) -``` - -This causes bundler issues because dynamic imports can't be properly tree-shaken, potentially pulling server-only code (like `Buffer`, `drizzle`, `postgres`) into the client bundle. - -✅ **Good - Static imports:** - -```typescript -// At the top of the route file -import { listRoles } from '~/utils/roles.server' - -// In component code -const rolesQuery = useQuery({ - queryFn: async () => { - return listRoles({ data: {} }) - }, -}) -``` - -Since `listRoles` is wrapped in `createServerFn`, TanStack Start will properly handle environment shaking and exclude server-only dependencies from the client bundle. - -### Rules for Server-Only Imports - -1. **Server functions** (`createServerFn` wrappers) can be imported statically anywhere -2. **Direct server-only code** (database clients, file system, etc.) must ONLY be imported: - - Inside `createServerFn` handlers - - In separate server-only files (e.g., `*.server.ts`) - - Never use dynamic imports (`await import()`) for server-only code in component code - -## Development & Build Commands - -### Don't Build After Every Change - -**Do not run builds after every change, especially for visual changes.** - -This is a visual website, not a library. Assume changes work unless the user reports otherwise. Running builds after every change wastes time and context. - -### Run Tests after Code Changes - -**After making code changes, always run `pnpm test` to verify the code passes basic tests.** - -Do NOT run tests after every tiny small change, just at the end of your known tasks. Tests are run automatically by the pre-commit hook and CI. Linting is fast and catches most issues immediately. - -### Debugging Visual Issues - -When the user reports something doesn't work or look right: - -1. Use the Playwright MCP to view the page and debug visually -2. Use builds (`pnpm build`) only when investigating build/bundler issues -3. Use linting (`pnpm lint`) to check for code issues - -### Use `build` for Build-Specific Issues - -**The `dev` command does not end, it runs indefinitely in watch mode.** - -Only use `build` when: - -- Investigating bundler or build-time errors -- Verifying production output -- The user specifically asks to verify the build - -### Testing with Playwright - -**Use the Playwright MCP for visual debugging and verification.** - -When debugging issues or verifying visual changes work correctly: - -- Navigate to the relevant page using Playwright -- Take snapshots or screenshots to verify the UI -- Interact with elements to test functionality - -This is the preferred method for verifying visual changes since this is a visual site. - -## UI Style Guide 2026 - -### Core Principles - -- Prioritize clarity, hierarchy, and calm -- Use depth to communicate structure, not decoration -- Favor warmth and approachability over stark minimalism - -### Layout - -- Prefer fewer, well defined containers over many small sections -- Use generous spacing to create separation before adding visual effects -- Cards are acceptable when they express grouping or hierarchy - -### Corners - -- Rounded corners are standard -- Use subtle radius values that feel intentional, not playful -- Avoid sharp 90 degree corners unless intentionally industrial - -### Shadows and Depth - -- Shadows should be soft, low contrast, and diffused -- Use shadows to imply separation, not elevation theatrics -- Avoid heavy drop shadows or strong directional lighting -- One to two shadow layers max - -### Cards - -- Cards should feel grounded, not floating -- Prefer light elevation, border plus shadow, or surface contrast -- Avoid overusing cards as a default layout primitive - -### Color and Surfaces - -- Favor soft neutrals, off whites, and warm grays -- Use surface contrast or translucency instead of strong outlines -- Glass or frosted effects are acceptable when subtle and accessible - -### Interaction - -- Use micro transitions to reinforce spatial relationships -- Hover and focus states should feel responsive, not animated -- Avoid excessive motion or springy effects - -### Typography - -- Let type hierarchy do most of the work -- Strong headings, calm body text -- Avoid visual noise around content - -### What to Avoid - -- Chunky shadows -- Overly flat, sterile layouts -- Neumorphism as a primary style -- Over designed card grids - -### Summary Rule - -**If depth does not improve comprehension, remove it.** - -## Writing Style - -### Avoid Emdashes - -**Never use emdashes (—) in writing. They make content smell like AI-generated text.** - -Use these alternatives instead: - -- Commas for parenthetical phrases -- Colons to introduce lists or explanations -- Periods to break into separate sentences -- Parentheses when appropriate - -❌ **Bad:** - -``` -RSCs aren't about replacing client interactivity — they're about choosing where work happens. -``` - -✅ **Good:** - -``` -RSCs aren't about replacing client interactivity. They're about choosing where work happens. -``` - -❌ **Bad:** - -``` -Heavy dependencies — markdown parsers, syntax highlighters — stay on the server. -``` - -✅ **Good:** - -``` -Heavy dependencies (markdown parsers, syntax highlighters) stay on the server. -``` diff --git a/AGENTS.md b/AGENTS.md new file mode 120000 index 000000000..49055565e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +.agents/index.md \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 000000000..49055565e --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +.agents/index.md \ No newline at end of file diff --git a/README.md b/README.md index 90cba4aac..f168f491e 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,19 @@ -# Welcome to TanStack.com! +
-This site is built with TanStack Router! +# TanStack.com -- [TanStack Router Docs](https://tanstack.com/router) +The home of the TanStack ecosystem. Built with [TanStack Router](https://tanstack.com/router) and deployed on [Cloudflare Workers](https://workers.cloudflare.com/). -It's deployed automagically with Netlify! +Follow @TanStack -- [Netlify](https://netlify.com/) +### [Become a Sponsor!](https://github.com/sponsors/tannerlinsley/) + +
## Development +### Quick Start + From your terminal: ```sh @@ -19,54 +23,64 @@ pnpm dev This starts your app in development mode, rebuilding assets on file changes. -## Editing and previewing the docs of TanStack projects locally - -The documentations for all TanStack projects except for `React Charts` are hosted on [https://tanstack.com](https://tanstack.com), powered by this TanStack Router app. -In production, the markdown doc pages are fetched from the GitHub repos of the projects, but in development they are read from the local file system. +### Local Setup -Follow these steps if you want to edit the doc pages of a project (in these steps we'll assume it's [`TanStack/form`](https://github.com/tanstack/form)) and preview them locally : +The documentation for all TanStack projects (except `React Charts`) is hosted on [tanstack.com](https://tanstack.com). In production, doc pages are fetched from GitHub. In development, they're read from your local file system. -1. Create a new directory called `tanstack`. +Create a `tanstack` parent directory and clone this repo alongside the projects: ```sh -mkdir tanstack +mkdir tanstack && cd tanstack +git clone git@github.com:TanStack/tanstack.com.git +git clone git@github.com:TanStack/query.git +git clone git@github.com:TanStack/router.git +git clone git@github.com:TanStack/table.git ``` -2. Enter the directory and clone this repo and the repo of the project there. +Your directory structure should look like this: -```sh -cd tanstack -git clone git@github.com:TanStack/tanstack.com.git -git clone git@github.com:TanStack/form.git ``` +tanstack/ + ├── tanstack.com/ + ├── query/ + ├── router/ + └── table/ +``` + +> [!WARNING] +> Directory names must match repo names exactly (e.g., `query` not `tanstack-query`). The app finds docs by looking for sibling directories by name. + +### Editing Docs + +To edit docs for a project, make changes in its `docs/` folder (e.g., `../form/docs/`) and visit http://localhost:3000/form/latest/docs/overview to preview. > [!NOTE] -> Your `tanstack` directory should look like this: -> -> ``` -> tanstack/ -> | -> +-- form/ -> | -> +-- tanstack.com/ -> ``` +> Updated pages need to be manually reloaded in the browser. > [!WARNING] -> Make sure the name of the directory in your local file system matches the name of the project's repo. For example, `tanstack/form` must be cloned into `form` (this is the default) instead of `some-other-name`, because that way, the doc pages won't be found. +> Update the project's `docs/config.json` if you add a new doc page! -3. Enter the `tanstack/tanstack.com` directory, install the dependencies and run the app in dev mode: +## Get Involved -```sh -cd tanstack.com -pnpm i -# The app will run on https://localhost:3000 by default -pnpm dev -``` +- We welcome issues and pull requests! +- Participate in [GitHub Discussions](https://github.com/TanStack/tanstack.com/discussions) +- Chat with the community on [Discord](https://discord.com/invite/WrRKjPJ) -4. Now you can visit http://localhost:3000/form/latest/docs/overview in the browser and see the changes you make in `tanstack/form/docs`. +## Explore the TanStack Ecosystem -> [!NOTE] -> The updated pages need to be manually reloaded in the browser. +- TanStack Config – Tooling for JS/TS packages +- TanStack DB – Reactive sync client store +- TanStack DevTools – Unified devtools panel +- TanStack Form – Type‑safe form state +- TanStack Pacer – Debouncing, throttling, batching +- TanStack Query – Async state & caching +- TanStack Ranger – Range & slider primitives +- TanStack Router – Type‑safe routing, caching & URL state +- TanStack Start – Full‑stack SSR & streaming +- TanStack Store – Reactive data store +- TanStack Table – Headless datagrids +- TanStack Virtual – Virtualized rendering -> [!WARNING] -> You will need to update the `docs/config.json` file (in the project's repo) if you add a new doc page! +… and more at TanStack.com » + + diff --git a/content-collections.ts b/content-collections.ts index 1ef0f4a10..ed3891941 100644 --- a/content-collections.ts +++ b/content-collections.ts @@ -1,34 +1,57 @@ import { defineCollection, defineConfig } from '@content-collections/core' -import { extractFrontMatter } from '~/utils/documents.server' +import { libraryIds } from '~/libraries/libraries' +import { normalizeRedirectFrom } from '~/utils/redirects' +import { z } from 'zod' + +const libraryIdSet = new Set(libraryIds) +const libraryListSchema = z.string().refine( + (value) => { + const libraries = value + .split(',') + .map((library) => library.trim()) + .filter(Boolean) + + return ( + libraries.length > 0 && + libraries.every((library) => libraryIdSet.has(library)) + ) + }, + { + message: `Expected comma-separated library ids: ${libraryIds.join(', ')}`, + }, +) const posts = defineCollection({ name: 'posts', directory: './src/blog', include: '*.md', - schema: (z) => ({ + schema: z.object({ title: z.string(), - published: z.string().date(), + published: z.iso.date(), draft: z.boolean().optional(), + excerpt: z.string(), authors: z.string().array(), + library: libraryListSchema.optional(), + content: z.string(), + redirect_from: z.string().array().optional(), }), transform: ({ content, ...post }) => { - const frontMatter = extractFrontMatter(content) - // Extract header image (first image after frontmatter) const headerImageMatch = content.match(/!\[([^\]]*)\]\(([^)]+)\)/) const headerImage = headerImageMatch ? headerImageMatch[2] : undefined + const redirectFrom = normalizeRedirectFrom(post.redirect_from) return { ...post, slug: post._meta.path, - excerpt: frontMatter.excerpt, - description: frontMatter.data.description, headerImage, + redirect_from: redirectFrom, + redirectFrom, content, } }, }) export default defineConfig({ - collections: [posts], + content: [posts], }) diff --git a/docs-info.md b/docs-info.md new file mode 100644 index 000000000..a13e6ef65 --- /dev/null +++ b/docs-info.md @@ -0,0 +1,272 @@ +# Maintainer Documentation Guide + +This page covers the markdown features supported in TanStack docs and the preferred workflow for future redirects. + +## Redirects + +For new page moves or consolidations, keep the canonical page and list old URLs in frontmatter with `redirect_from`. + +```yaml +--- +title: Overview +redirect_from: + - /framework/react/overview + - /framework/solid/overview +--- +``` + +In the example above, old framework-specific URLs will redirect to the shared `/overview` page without needing to add a new entry to the central redirect files in the `tanstack.com` repo. + +## Supported markdown + +Docs support normal GitHub-flavored markdown, including: + +- headings +- links +- lists +- tables +- fenced code blocks +- images +- blockquotes +- task lists + +## Callouts + +GitHub-style alerts are supported. For a customized title, for example to replace `Note` with something else, you can use the syntax `> [!TYPE] Title`: + +```md +> [!NOTE] Custom title +> Use `redirect_from` on the canonical page when docs are moved. + +> [!TIP] +> Prefer absolute paths like `/framework/react/overview`. +``` + +## Code Blocks + +Fenced code blocks are supported with language identifiers for syntax highlighting. You can also add metadata like `title="..."` for file tabs. + +````md +```tsx title="app.tsx" +export function App() { + return
Hello
+} +``` +```` + +## Tabs component + +The tabs component lets you group content into tabbed sections. It supports multiple variants, including file tabs and package manager tabs. + +### File tabs + +Use `variant="files"` when the block should render code examples as file tabs. It scans consecutive code blocks, extracts language, title, and content, and uses that to build `file-tab` data. Titles come from code-block metadata such as `title="..."` or will default to the language name if not provided. + +````md + + +```tsx file="app.tsx" +export function App() { + return
Hello
+} +``` + +```css file="styles.css" +body { + color: tomato; +} +``` + + +```` + +### What matters + +- use fenced code blocks +- add `title="..."` if you want meaningful file tab labels +- language comes from the code fence language +- code text is extracted from the `` node content + +### Package manager tabs + +Package-manager tabs are a special `tabs` variant. The parser reads framework lines like `react: ...` or `solid: ...`, groups packages, and later generates package-manager-specific commands. + +There are various supported package manager formats, including npm, yarn, pnpm, and bun. + +If you're looking to support a multi-line command, you can add multiple instances of the same framework. For example: + +```md + + +react: +react: + + +``` + +This will become: + +```bash +npm i +npm i +``` + +#### Supported modes + +- `install` (default) +- `dev-install` +- `local-install` +- `create` +- `custom` (for custom command templates) + +##### Install (default) + +```md + + +react: +solid: + + +``` + +becomes + +```bash +npm i +``` + +##### Dev install + +```md + + +react: +solid: + + +``` + +becomes + +```bash +npm i -D +``` + +##### Local install + +```md + + +react: +solid: + + +``` + +becomes + +```bash +npx --workspace=./path/to/workspace +``` + +##### Create + +```md + + +react: +solid: + + +``` + +becomes + +```bash +npm create +``` + +##### Custom + +```md + + +react: +solid: + + +``` + +becomes + +```bash +npm +``` + +### Bundler tabs + +Bundler tabs render a compact tab row (like package-manager tabs) but accept rich markdown content per bundler (like the framework component). The user's bundler choice is persisted to `localStorage` and synced across every bundler tab block on the page. + +Inside `variant="bundler"`, each top-level heading whose text matches a known bundler starts a new section, and the following nodes (prose, code blocks, etc.) become that bundler's panel. The transformer uses the largest heading level present in the block, so `# Vite` / `# Rsbuild` and `## Vite` / `## Rsbuild` both work — just be consistent within a single block. + +````md + + +# Vite + +```ts title="vite.config.ts" +import { defineConfig } from 'vite' + +export default defineConfig({}) +``` + +# Rsbuild + +```ts title="rsbuild.config.ts" +import { defineConfig } from '@rsbuild/cli' + +export default defineConfig({}) +``` + + +```` + +Supported bundlers: `vite`, `rsbuild`. Heading text is matched case-insensitively. Both sections should be defined; if the user's selected bundler isn't present in a particular block, the first defined panel is shown as a fallback. + +## Framework component + +Framework blocks let one markdown source contain React, Solid, or other framework-specific content. Internally, the transformer looks for h1 headings inside the framework block and treats each `# Heading` as a framework section boundary. It then stores framework metadata and rewrites the block into separate framework panels. + +> **Note**: This should only be used when the majority of the content is the same. If the content is mostly different, it's better to create separate markdown files for each framework and use redirects to point to the canonical page. + +````md + + +# React + +Use the React adapter here. + +```tsx +// React code +``` + +# Solid + +Use the Solid adapter here. + +```tsx +// Solid Code +``` + + +```` + +Each top-level `#` heading becomes a framework panel. Nested headings inside a framework section are preserved for the table of contents. + +## Editing notes + +- Keep redirects on the surviving page, not on the page being removed. +- Use absolute paths in `redirect_from`. +- Avoid duplicate `redirect_from` values across pages. +- Existing central redirect files still handle older redirects; use frontmatter for new ones going forward. diff --git a/docs/cloudflare-workers-migration.md b/docs/cloudflare-workers-migration.md new file mode 100644 index 000000000..006beca63 --- /dev/null +++ b/docs/cloudflare-workers-migration.md @@ -0,0 +1,135 @@ +# Cloudflare Workers Migration Report + +TanStack.com is configured as a Cloudflare Workers deployment for this branch. Netlify is no longer part of this site's hosting configuration, but Netlify remains available in builder/deploy-provider UX for users who choose it. + +## Files Changed + +- `package.json`, `pnpm-lock.yaml`: Cloudflare build/preview/deploy scripts, `@cloudflare/vite-plugin`, `wrangler`, `@tanstack/create@0.68.4`, and removal of Netlify hosting packages. +- `vite.config.ts`: Cloudflare Vite plugin, Worker build constants, opt-in image transformation flag, and server builder-generation enabled with the Worker-safe create API. +- `wrangler.jsonc`: Worker name/account, assets binding, `nodejs_compat`, CPU limit, cron triggers, and production `SITE_URL`. +- `src/server.ts`, `src/server/scheduled.server.ts`: Worker `fetch` and `scheduled` entrypoints, replacing former Netlify scheduled functions. +- `src/server/runtime/host.server.ts`, `src/utils/hosting-cache.server.ts`: host cache purge adapter using Cloudflare cache-tag purge. +- `src/components/OptimizedImage.tsx`, `src/utils/optimizedImage.ts`: host-neutral optimized image helper with Cloudflare image transformations behind an explicit build flag. +- `src/builder/api/create-worker.ts`: host-neutral adapter around `@tanstack/create/worker` with local Worker manifest loaders for supported frameworks and add-ons. +- `src/builder/api/*`, `src/routes/api/builder/*`: builder feature catalog, compile, download, validate, suggest, and GitHub deploy paths use the Worker-safe create adapter. +- `src/routes/*`, `src/utils/*`, `src/server/*`: CDN cache headers moved from Netlify-specific headers to Cloudflare Workers Cache headers / `Cache-Tag`. +- `src/utils/markdown/processor.ts`: site-side compatibility guard for escaped angle brackets in generated TypeDoc markdown until `@tanstack/markdown` handles `\<...\>` as escaped text. +- `package.json`, `pnpm-lock.yaml`: `@tanstack/markdown@0.0.5` for compact table delimiters, footnotes, and legacy single-tilde strike headings without the temporary pnpm patch. +- Removed hosting-only Netlify files: `netlify.toml`, `netlify/functions/*`, `scripts/run-built-server.mjs`. + +## Commands Used + +```bash +pnpm install --lockfile-only +pnpm add @tanstack/create@^0.68.4 +pnpm run test:tsc +pnpm exec tsc --noEmit +pnpm run build:cloudflare +pnpm test +pnpm run dev:cloudflare +pnpm run deploy:cloudflare +pnpm run preview:cloudflare -- --host 127.0.0.1 --port 3001 +pnpm install +``` + +Additional checks used `curl`, Node fetch scripts, Wrangler tail, and Playwright with system Chrome against the Workers preview URL. + +## Deploy Cache Purge + +`pnpm run deploy:cloudflare` builds, deploys the Worker, then runs `pnpm run purge:cloudflare`. The purge step clears the Cloudflare zone cache with `purge_everything` so stale HTML documents cannot keep pointing at removed route chunks after a deploy. + +Required environment: + +- `CLOUDFLARE_ZONE_ID`: the active `tanstack.com` zone ID. +- `CLOUDFLARE_CACHE_PURGE_TOKEN`: Cloudflare API token with `Cache Purge` permission for that zone. + +`CLOUDFLARE_API_TOKEN` / `CF_API_TOKEN` and `CF_ZONE_ID` are accepted as aliases. If `CLOUDFLARE_ZONE_ID` is missing, the script can discover it only when the token also has zone read access. + +## Worker + +- Account: `8da95258a9c70b54c3e2b374a0079106` +- Worker: `tanstack-com` +- URL: `https://tanstack-com.thetanstack.workers.dev` +- Current version: `ff011a60-320b-43b7-9a03-bd650a41bc7b` +- Upload size: `17748.26 KiB` raw, `6413.10 KiB` gzip +- Startup time: `33 ms` +- Note: the secret-bearing `tanstack-com-staging` Worker was renamed to `tanstack-com`, and the older empty `tanstack-com` Worker was removed. + +## Passed + +- `pnpm run build:cloudflare` passed. +- `pnpm run test:tsc` passed. +- `pnpm test` passed with 10 existing oxlint warnings. +- Cloudflare deploy passed. +- Local Cloudflare preview started and returned 200 for `/` and `/builder`. +- `/` returned 200 HTML on the Worker. +- `/start/latest` returned 200 HTML on the Worker. +- Browser SPA navigation from `/` to `/start/latest` did not reproduce the `npm-recent-downloads ... data is undefined` error. +- `/builder` returned 200 HTML with COOP/COEP headers and loaded the client builder/integration surface. +- Primary homepage images loaded in browser on the Worker. +- `/api/og/query.png?title=Query&description=Smoke` returned a valid 1200x630 PNG. +- `/_a/gtag.js` returned Google's JavaScript. +- `/_a/g/collect` returned 204. +- `/auth/github/start?returnTo=/account` redirected to GitHub with secure state/return cookies. +- `/.well-known/oauth-authorization-server` returned OAuth metadata. +- `/api/mcp/` returned the expected unauthenticated JSON-RPC auth error instead of a runtime failure. +- `POST /api/application-starter/resolve` returned a Start recipe. +- `/api/builder/features?framework=react` returned the Worker-safe catalog, including `tanstack-query`, `cloudflare`, and `biome`. +- `/api/builder/compile` generated representative React and Solid projects server-side on the Worker. +- `/api/builder/download` returned a valid zip for a representative React builder project. +- `https://tanstack.com/api/builder/features?framework=react` returned the same Worker-safe builder catalog. +- `https://tanstack.com/builder` returned 200 HTML with COOP/COEP headers. +- Worker output does not include `generated/create-manifest.js` or a `create-manifest` chunk. +- Cloudflare dry-run/upload size stayed below the paid Worker 10 MiB gzip limit after excluding the heavy React `events` example implementation chunk. +- `Link` response headers for static assets are emitted on SSR responses for Cloudflare Early Hints fallback. +- Broad docs/blog audit generated 2,767 latest-doc/blog URLs from GitHub doc trees plus local blog posts and compared production vs Worker. +- Escaped generic headings in TypeDoc markdown now render correctly, e.g. `Interface: AudioAdapter` with the production-compatible `interface-audioadaptertmodel-tprovideroptions` anchor. +- Local site parser checks now match the remaining markdown audit diffs: + - `/ai/latest/docs/code-mode/code-mode` parses 4 content tables. + - `/db/latest/docs/collections/powersync-collection` parses 24 content tables. + - `/start/latest/docs/framework/react/migrate-from-next-js` headings render as `Server Actions Functions` and `Server Routes Handlers`. + - `/blog/tanstack-router-signal-graph` renders footnotes with `data-footnotes` and production-compatible footnote anchors. +- Live Worker rechecks against production passed for the same table, heading, and footnote signals after deploying version `586b99ec-4a2c-46d2-b3b3-eb775de13141`. +- Three full-body rechecks of 43 URLs that intermittently returned Worker 500/timeout during the high-concurrency audit cleared; the only stable non-200s were `/hotkeys/latest/docs/reference` and `/pacer/latest/docs/reference`, both 404 on production and Worker. + +## Failed Or Not Proven + +- Full GitHub OAuth callback/account login was not completed. +- End-to-end GitHub repository deploy was not completed with a logged-in account. +- Cron trigger behavior was deployed but not manually invoked. +- The React `events` example is not exposed in the Worker builder catalog because its implementation chunk pushes the Worker over Cloudflare's paid 10 MiB gzip upload limit. +- High-concurrency audit runs can still produce transient Worker 500s with `{"status":500,"unhandled":true,"message":"HTTPError"}` on changing docs paths, but targeted full-body rechecks did not reproduce stable page failures. Treat this as a load/audit-cache risk, not a confirmed content regression. + +## Builder Generation Note + +`@tanstack/create@0.68.4` adds the lazy `@tanstack/create/worker` API. The site now imports that API through `src/builder/api/create-worker.ts` instead of importing `@tanstack/create/edge` from route logic. + +`/api/builder/features` remains catalog-only by using `create.getFrameworkById` and `create.getAllAddOns`. ZIP/project generation loads only the requested framework/add-on chunks, then calls `create.finalizeAddOns`, `create.populateAddOnOptionsDefaults`, `createMemoryEnvironment`, and `create.createApp`. + +The Worker build was checked for `generated/create-manifest.js` and `create-manifest` output, and no generated manifest bundle was present. Including the React `events` example implementation still pushed upload size to `11179.57 KiB` gzip, so the Worker loader intentionally omits that chunk. The deployed Worker is `6413.10 KiB` gzip. + +## Image Transformation Note + +Cloudflare image transformations are enabled for `build:cloudflare`. Transformed image URLs are emitted through the configured `SITE_URL` origin because `/cdn-cgi/image/*` works on the routed `tanstack.com` zone but still returns 404 on the `workers.dev` hostname. + +## Markdown Audit Note + +The new markdown renderer initially parsed escaped TypeDoc generics like `\` as inline HTML. The site now protects escaped `<` / `>` outside code fences and inline code before parsing, restores them into text nodes before render, and rebuilds headings so rendered content and ToC anchors stay aligned. + +Resolved markdown differences from the broad audit: + +- Escaped TypeDoc generics no longer become inline HTML. +- Compact markdown table delimiters like `:--:` now parse as tables, matching the existing production renderer. +- Blog footnotes now render with `data-footnotes`, `user-content-fn-*`, and `user-content-fnref-*` anchors. +- Legacy single-tilde strike headings now render without visible `~` markers. + +Remaining markdown differences observed during audit: + +- Production duplicates light/dark code blocks; the Worker branch renders one theme-aware code block. This explains large HTML-size and `
` count differences.
+- The temporary `@tanstack/markdown@0.0.4` pnpm patch has been removed after upgrading to `@tanstack/markdown@0.0.5`.
+
+## Readiness
+
+Core marketing SSR, docs/start navigation, security headers, static assets, analytics proxying, GitHub auth start, MCP auth rejection, application-starter API, scheduled Worker registration, Cloudflare preview, deploy, dynamic OG image generation, and representative Worker-side builder generation are working on Cloudflare Workers.
+
+Production migration is close, but not fully safe until logged-in OAuth/account flows pass, cron jobs are verified, and an authenticated builder GitHub deploy is completed. The main remaining builder gap is the omitted React `events` example chunk.
diff --git a/docs/ecosystem-implementation-todo.md b/docs/ecosystem-implementation-todo.md
new file mode 100644
index 000000000..8383b310a
--- /dev/null
+++ b/docs/ecosystem-implementation-todo.md
@@ -0,0 +1,145 @@
+# Ecosystem Implementation Todo
+
+## Goal
+
+Build a database-driven `/ecosystem` marketplace and submission flow without blocking the partner-page SEO work.
+
+## Product Goals
+
+- Create an `/ecosystem` directory page that acts like a marketplace
+- Let users sort and filter entries by the fields that actually matter
+- Let companies submit new listings through a first-party submission flow
+- Seed the marketplace with existing TanStack partners when the DB work starts
+- Let TanStack maintainers add an official review and comparisons to each listing
+- Keep partner-backed entries aligned with `/partners/[partner]` pages
+
+## Scope To Build Later
+
+### Data model
+
+- Add a new `ecosystemEntries` table
+- Add a dedicated moderation capability like `moderate-ecosystem`
+- Keep the schema intentionally small and high-value
+
+Suggested v1 fields:
+
+- `id`
+- `userId`
+- `slug`
+- `name`
+- `websiteUrl`
+- `kind`
+- `tags`
+- `libraries`
+- `pricingModel`
+- `isOpenSource`
+- `preferenceScore`
+- `isPartner`
+- `logoUrl`
+- `screenshotUrl`
+- `summary`
+- `officialReviewMd`
+- `competitorSlugs`
+- `status`
+- `moderationNote`
+- `moderatedBy`
+- `moderatedAt`
+- `createdAt`
+- `updatedAt`
+
+### Taxonomy
+
+Use a very small taxonomy surface:
+
+- `kind`: `integration | service | tool | template | plugin`
+- `pricingModel`: `free | freemium | paid | contact | oss`
+- `tags: string[]` absorbs both "type" and "service"
+
+### Comparison model
+
+Use slugs everywhere.
+
+- Partners compare against other partner ids, which are also their slugs
+- Ecosystem entries compare against ecosystem slugs
+- Partner-backed ecosystem entries should reuse the same slug namespace
+
+Suggested v1 field:
+
+- `competitorSlugs: string[]`
+
+No UUID-based comparison references for v1.
+No partner-only comparison system.
+No extra comparison tables unless the simple shape breaks down.
+
+### Canonical URL rules
+
+- Partner-backed entries should be canonical at `/partners/[partner]`
+- Non-partner entries should be canonical at `/ecosystem/[slug]`
+- If a partner-backed entry is reachable from `/ecosystem/[slug]`, redirect or canonicalize to `/partners/[partner]`
+
+### Submission flow
+
+Build a user-facing submission portal similar in spirit to showcase, but for ecosystem listings.
+
+Suggested v1 required fields:
+
+- `name`
+- `websiteUrl`
+- `kind`
+- `tags`
+- `libraries`
+- `pricingModel`
+- `isOpenSource`
+- `logoUrl`
+- `screenshotUrl`
+- `summary`
+
+Maintainer-only fields for v1:
+
+- `preferenceScore`
+- `officialReviewMd`
+- `competitorSlugs`
+- moderation fields
+
+### Admin flow
+
+- Add `/admin/ecosystem`
+- Add `/admin/ecosystem/[id]`
+- Let maintainers moderate submissions
+- Let maintainers edit all listing fields
+- Let maintainers set `preferenceScore`
+- Let maintainers author `officialReviewMd`
+- Let maintainers manage `competitorSlugs`
+
+### Account flow
+
+- Add `/account/ecosystem`
+- Let users view, edit, and delete their own submissions
+- Keep this separate from `/account/integrations`, which already means something else
+
+## Seeding Plan
+
+- Seed existing partners into the ecosystem table when the DB work starts
+- Preserve current partner ids as ecosystem slugs for partner-backed entries
+- Seed `preferenceScore` from the existing `partners[].score` values
+- Start seeded partner entries as `approved`
+- Keep the current score scale and revisit later if needed
+
+## Suggested Implementation Order
+
+1. Add ecosystem capability, enums, and DB schema
+2. Add ecosystem server functions and query options
+3. Seed partner rows into the new table
+4. Build `/ecosystem` index and `/ecosystem/[slug]`
+5. Build `/ecosystem/submit` and edit flows
+6. Build `/account/ecosystem`
+7. Build `/admin/ecosystem`
+8. Add SEO metadata and JSON-LD
+9. Run `pnpm test`
+
+## Notes
+
+- Reuse showcase architecture patterns, not the showcase schema
+- Keep the implementation intentionally small at first
+- Avoid over-modeling comparison data until real usage demands it
+- Partner pages ship first; ecosystem follows later
diff --git a/docs/mcp/config.json b/docs/mcp/config.json
deleted file mode 100644
index 945453f92..000000000
--- a/docs/mcp/config.json
+++ /dev/null
@@ -1,21 +0,0 @@
-{
-  "$schema": "https://tanstack.com/tanstack-docs-config.schema.json",
-  "docSearch": {
-    "appId": "FQ0DQ6MA3C",
-    "apiKey": "10c34d6a5c89f6048cf644d601e65172",
-    "indexName": "tanstack-test"
-  },
-  "sections": [
-    {
-      "label": "Getting Started",
-      "children": [
-        { "label": "Overview", "to": "overview" },
-        { "label": "Connecting", "to": "connecting" }
-      ]
-    },
-    {
-      "label": "Reference",
-      "children": [{ "label": "Available Tools", "to": "tools" }]
-    }
-  ]
-}
diff --git a/docs/mcp/connecting.md b/docs/mcp/connecting.md
deleted file mode 100644
index 7eca1b683..000000000
--- a/docs/mcp/connecting.md
+++ /dev/null
@@ -1,296 +0,0 @@
----
-id: connecting
-title: Connecting
----
-
-The TanStack MCP Server is available at `https://tanstack.com/api/mcp` and uses the Streamable HTTP transport.
-
-## Authentication Options
-
-There are two ways to authenticate with the TanStack MCP server:
-
-### OAuth (Recommended)
-
-MCP clients that support OAuth can authenticate automatically. Just use the server URL and your client will open a browser window to authorize access:
-
-```
-https://tanstack.com/api/mcp
-```
-
-No API key needed. Your client handles the OAuth flow automatically.
-
-### API Keys
-
-For clients that don't support OAuth, or if you prefer manual key management:
-
-1. [Sign in to your TanStack account](/login)
-2. Go to [Integrations](/account/integrations)
-3. Click "New Key" and give it a descriptive name
-4. Copy the key immediately (you won't see it again)
-
-> [!NOTE]
-> Replace `YOUR_API_KEY` in the examples below with your actual API key.
-
-## Claude Desktop
-
-Claude Desktop supports OAuth authentication. Add to your config:
-
-**macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`
-**Windows:** `%APPDATA%\Claude\claude_desktop_config.json`
-
-```json
-{
-  "mcpServers": {
-    "tanstack": {
-      "url": "https://tanstack.com/api/mcp"
-    }
-  }
-}
-```
-
-Restart Claude Desktop. On first use, a browser window will open to authorize access.
-
-
-Using an API key instead - -```json -{ - "mcpServers": { - "tanstack": { - "command": "npx", - "args": ["mcp-remote", "https://tanstack.com/api/mcp"], - "env": { - "MCP_HEADERS": "Authorization: Bearer YOUR_API_KEY" - } - } - } -} -``` - -
- -## Claude Code - -```bash -claude mcp add --transport http tanstack https://tanstack.com/api/mcp -``` - -On first use, a browser window will open to authorize access. - -
-Using an API key instead - -```bash -claude mcp add --transport http tanstack https://tanstack.com/api/mcp --header "Authorization: Bearer YOUR_API_KEY" -``` - -
- -## Cursor - -Add to your Cursor MCP configuration: - -```json -{ - "mcpServers": { - "tanstack": { - "url": "https://tanstack.com/api/mcp" - } - } -} -``` - -
-Using an API key instead - -```json -{ - "mcpServers": { - "tanstack": { - "command": "npx", - "args": ["mcp-remote", "https://tanstack.com/api/mcp"], - "env": { - "MCP_HEADERS": "Authorization: Bearer YOUR_API_KEY" - } - } - } -} -``` - -
- -## VS Code - -Add to your VS Code settings or `.vscode/mcp.json`: - -```json -{ - "servers": { - "tanstack": { - "url": "https://tanstack.com/api/mcp" - } - } -} -``` - -
-Using an API key instead - -```json -{ - "servers": { - "tanstack": { - "type": "http", - "url": "https://tanstack.com/api/mcp", - "headers": { - "Authorization": "Bearer YOUR_API_KEY" - } - } - } -} -``` - -
- -## OpenCode - -Add to your OpenCode MCP configuration in `~/.config/opencode/config.json`: - -```json -{ - "mcp": { - "servers": { - "tanstack": { - "type": "remote", - "url": "https://tanstack.com/api/mcp" - } - } - } -} -``` - -After adding the config, run the auth flow manually the first time: - -```bash -npx mcp-remote auth https://tanstack.com/api/mcp -``` - -This opens a browser to authorize access. After that, OpenCode will connect automatically. - -
-Using an API key instead - -```json -{ - "mcp": { - "servers": { - "tanstack": { - "type": "remote", - "url": "https://tanstack.com/api/mcp", - "headers": { - "Authorization": "Bearer YOUR_API_KEY" - } - } - } - } -} -``` - -
- -## Windsurf - -Add to your Windsurf MCP configuration: - -```json -{ - "mcpServers": { - "tanstack": { - "serverUrl": "https://tanstack.com/api/mcp" - } - } -} -``` - -
-Using an API key instead - -```json -{ - "mcpServers": { - "tanstack": { - "serverUrl": "https://tanstack.com/api/mcp", - "headers": { - "Authorization": "Bearer YOUR_API_KEY" - } - } - } -} -``` - -
- -## MCP Inspector - -Use the MCP Inspector to test the server interactively: - -```bash -npx @modelcontextprotocol/inspector https://tanstack.com/api/mcp -``` - -
-Using an API key instead - -```bash -npx @modelcontextprotocol/inspector https://tanstack.com/api/mcp --header "Authorization: Bearer YOUR_API_KEY" -``` - -
- -## Custom Integration - -For custom integrations, the server accepts standard MCP requests via HTTP: - -- **Endpoint:** `https://tanstack.com/api/mcp` -- **Transport:** Streamable HTTP (stateless) -- **Methods:** POST (for requests), GET (for server-sent events), DELETE (for session cleanup) -- **Authentication:** OAuth 2.1 (recommended) or Bearer token via Authorization header - -### OAuth 2.1 Flow - -The server supports OAuth 2.1 with PKCE for secure authentication: - -1. Discover endpoints via `/.well-known/oauth-authorization-server` -2. Redirect user to `/oauth/authorize` with PKCE challenge -3. Exchange authorization code at `/oauth/token` -4. Use the access token as a Bearer token - -### API Key Authentication - -Example request using curl with an API key: - -```bash -curl -X POST https://tanstack.com/api/mcp \ - -H "Content-Type: application/json" \ - -H "Authorization: Bearer YOUR_API_KEY" \ - -d '{ - "jsonrpc": "2.0", - "id": 1, - "method": "tools/list" - }' -``` - -## Rate Limits - -API requests are rate-limited to protect the service: - -- **Authenticated requests:** 60 requests per minute -- Rate limit headers are included in responses (`X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`) - -## Verifying Connection - -After connecting, ask your AI assistant to list TanStack libraries: - -> "Use the TanStack MCP to list all available libraries" - -You should see a list of all TanStack libraries with their versions and descriptions. diff --git a/docs/mcp/overview.md b/docs/mcp/overview.md deleted file mode 100644 index c41d7f9f9..000000000 --- a/docs/mcp/overview.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -id: overview -title: Overview ---- - -TanStack MCP Server provides AI assistants with direct access to TanStack documentation through the [Model Context Protocol](https://modelcontextprotocol.io). This allows AI tools like Claude, Cursor, and others to search and retrieve up-to-date documentation for all TanStack libraries. - -## Why Use the MCP Server? - -AI assistants are trained on snapshots of documentation that become stale over time. The TanStack MCP Server solves this by giving AI tools real-time access to: - -- **Current documentation** for all TanStack libraries (Router, Query, Table, Form, Virtual, Store, and more) -- **Version-specific docs** for the exact version you're using -- **Full-text search** across all documentation - -## Quick Start - -Most MCP clients support OAuth authentication. Just add the server URL: - -``` -https://tanstack.com/api/mcp -``` - -Your client will open a browser window to authorize access on first use. No API key needed. - -See [Connecting](./connecting) for client-specific setup instructions. - -## Available Tools - -The MCP server exposes tools for documentation and showcase management: - -### Documentation Tools - -| Tool | Description | Auth Required | -| ---------------- | ------------------------------------------------------------ | ------------- | -| `list_libraries` | List all TanStack libraries with their versions and metadata | No | -| `get_doc` | Fetch a specific documentation page by library and path | No | -| `search_docs` | Full-text search across all TanStack documentation | No | - -### Showcase Tools - -| Tool | Description | Auth Required | -| ------------------- | ------------------------------------------ | ------------- | -| `search_showcases` | Search approved TanStack showcase projects | No | -| `get_showcase` | Get details of a specific showcase project | No | -| `submit_showcase` | Submit a new project to the showcase | Yes | -| `update_showcase` | Update your existing showcase submission | Yes | -| `delete_showcase` | Delete your showcase submission | Yes | -| `list_my_showcases` | List your own showcase submissions | Yes | - -### NPM Stats Tools - -| Tool | Description | Auth Required | -| --------------------------- | ------------------------------------------------------- | ------------- | -| `get_npm_stats` | Get aggregated download stats for TanStack or a library | No | -| `list_npm_comparisons` | List preset package comparisons (Data Fetching, etc.) | No | -| `compare_npm_packages` | Compare download stats for multiple packages over time | No | -| `get_npm_package_downloads` | Get detailed historical downloads for a single package | No | - -See [Available Tools](./tools) for detailed parameter documentation. - -## How It Works - -The TanStack MCP Server uses the Streamable HTTP transport, making it compatible with serverless deployments and any MCP client that supports HTTP transport. When your AI assistant needs TanStack documentation, it calls the appropriate tool and receives the content directly. - -``` -AI Assistant → MCP Client → TanStack MCP Server → Documentation -``` - -All documentation is fetched directly from the TanStack GitHub repositories, ensuring you always get the most current content for your specified version. diff --git a/docs/mcp/tools.md b/docs/mcp/tools.md deleted file mode 100644 index 8072d9141..000000000 --- a/docs/mcp/tools.md +++ /dev/null @@ -1,286 +0,0 @@ ---- -id: tools -title: Available Tools ---- - -The TanStack MCP Server provides 5 tools for accessing documentation, NPM statistics, and ecosystem recommendations. - -## list_libraries - -List all available TanStack libraries with their metadata. - -### Parameters - -| Parameter | Type | Required | Description | -| --------- | ------ | -------- | ------------------------------------------------------------------- | -| `group` | string | No | Filter by group: `state`, `headlessUI`, `performance`, or `tooling` | - -### Response - -- `group` - The group name or "All Libraries" -- `count` - Number of libraries returned -- `libraries` - Array of library objects with `id`, `name`, `tagline`, `description`, `frameworks`, `latestVersion`, `docsUrl`, and `githubUrl` - -### Example - -```json -{ - "name": "list_libraries", - "arguments": { - "group": "state" - } -} -``` - ---- - -## doc - -Fetch a specific documentation page by library and path. - -### Parameters - -| Parameter | Type | Required | Description | -| --------- | ------ | -------- | --------------------------------------------------------------------------------------- | -| `library` | string | Yes | Library ID (e.g., `query`, `router`, `table`, `form`) | -| `path` | string | Yes | Documentation path (e.g., `framework/react/overview`, `framework/react/guides/queries`) | -| `version` | string | No | Version (e.g., `v5`, `v1`). Defaults to `latest` | - -### Response - -- `title` - Document title from frontmatter -- `content` - Full markdown content of the document -- `url` - Canonical URL to the documentation page -- `library` - Library name -- `version` - Resolved version - -### Example - -```json -{ - "name": "doc", - "arguments": { - "library": "query", - "path": "framework/react/guides/queries", - "version": "v5" - } -} -``` - -### Common Documentation Paths - -Most TanStack libraries follow this path structure: - -- `framework/{framework}/overview` - Framework-specific overview -- `framework/{framework}/installation` - Installation guide -- `framework/{framework}/guides/{topic}` - Guides for specific topics -- `framework/{framework}/reference/{api}` - API reference - -Where `{framework}` is one of: `react`, `vue`, `solid`, `svelte`, `angular` - ---- - -## search_docs - -Full-text search across all TanStack documentation. - -### Parameters - -| Parameter | Type | Required | Description | -| ----------- | ------ | -------- | ------------------------------------------------------------ | -| `query` | string | Yes | Search query | -| `library` | string | No | Filter to specific library (e.g., `query`, `router`) | -| `framework` | string | No | Filter to specific framework (e.g., `react`, `vue`, `solid`) | -| `limit` | number | No | Maximum results (default: 10, max: 50) | - -### Response - -- `query` - The search query -- `totalHits` - Total number of matching documents -- `results` - Array of search results with `title`, `url`, `snippet`, `library`, and `breadcrumb` - -### Example - -```json -{ - "name": "search_docs", - "arguments": { - "query": "useQuery refetch", - "library": "query", - "framework": "react", - "limit": 5 - } -} -``` - ---- - -## npm_stats - -NPM download statistics for TanStack org, individual libraries, or package comparisons. - -### Parameters - -| Parameter | Type | Required | Description | -| ------------- | -------- | -------- | ------------------------------------------------------------------- | -| `library` | string | No | TanStack library ID (e.g., `query`, `router`) | -| `packages` | string[] | No | NPM packages to compare (max 10) | -| `preset` | string | No | Preset comparison ID (e.g., `data-fetching`) | -| `listPresets` | boolean | No | List available preset comparisons | -| `range` | string | No | Time range: `30d`, `90d`, `180d`, `1y`, `2y`, `all`. Default: `90d` | -| `bin` | string | No | Aggregation: `monthly`, `weekly`, `daily`. Default: `monthly` | - -### Modes - -The tool operates in different modes based on which parameters are provided: - -1. **No parameters** - Returns TanStack org summary with library breakdown -2. **`library`** - Returns stats for a specific TanStack library with package details -3. **`packages`** - Compares arbitrary NPM packages -4. **`preset`** - Uses a preset comparison (e.g., `data-fetching`, `state-management`) -5. **`listPresets: true`** - Lists all available presets - -### Response Examples - -**Org summary (no params):** - -```json -{ - "org": "tanstack", - "totalDownloads": 1234567890, - "ratePerDay": 5000000, - "libraries": [{ "id": "query", "downloads": 500000000, "packages": 12 }] -} -``` - -**Package comparison:** - -```json -{ - "range": { "start": "2024-01-01", "end": "2024-03-31" }, - "bin": "monthly", - "packages": [ - { - "name": "@tanstack/react-query", - "total": 15000000, - "avgPerDay": 165000, - "data": [{ "date": "2024-01-01", "downloads": 5000000 }] - } - ] -} -``` - -### Examples - -```json -// Org summary -{ "name": "npm_stats", "arguments": {} } - -// Library breakdown -{ "name": "npm_stats", "arguments": { "library": "query" } } - -// Compare packages -{ - "name": "npm_stats", - "arguments": { - "packages": ["@tanstack/react-query", "swr", "@apollo/client"], - "range": "1y", - "bin": "monthly" - } -} - -// Use preset -{ "name": "npm_stats", "arguments": { "preset": "data-fetching" } } - -// List presets -{ "name": "npm_stats", "arguments": { "listPresets": true } } -``` - -### Available Presets - -Use `listPresets: true` to see all available presets. Common ones include: - -- `data-fetching` - React Query, SWR, Apollo, tRPC -- `state-management` - Redux, MobX, Zustand, Jotai, Valtio -- `routing-react` - React Router, TanStack Router, Next, Wouter -- `data-grids` - AG Grid, TanStack Table, Handsontable -- `virtualization` - TanStack Virtual, react-window, react-virtuoso -- `frameworks` - React, Vue, Angular, Svelte, Solid -- `forms` - React Hook Form, TanStack Form, Conform -- `validation` - Zod, Yup, Valibot, ArkType - ---- - -## ecosystem - -Ecosystem partner recommendations filtered by category or TanStack library. - -### Parameters - -| Parameter | Type | Required | Description | -| ---------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------ | -| `category` | string | No | Filter by category: `database`, `auth`, `deployment`, `monitoring`, `cms`, `api`, `data-grid`, `code-review`, `learning` | -| `library` | string | No | Filter by TanStack library (e.g., `start`, `router`, `query`) | - -### Response - -- `query` - The filters applied -- `count` - Number of partners returned -- `partners` - Array of partners with `id`, `name`, `tagline`, `description`, `category`, `categoryLabel`, `url`, `libraries` - -### Example - -```json -{ - "name": "ecosystem", - "arguments": { - "category": "database", - "library": "start" - } -} -``` - ---- - -## Usage Tips - -### Finding Documentation - -1. **Start with `list_libraries`** to discover available libraries and their IDs -2. **Use `search_docs`** to find specific topics when you don't know the exact path -3. **Use `doc`** to fetch the full content once you know the path - -### Version Handling - -- Use `latest` (default) to always get the current documentation -- Specify a version like `v5` or `v4` when working with a specific library version -- The `list_libraries` tool shows available versions for each library - -### Path Discovery - -Search results include URLs that reveal the documentation path structure. For example, a URL like `https://tanstack.com/query/latest/docs/framework/react/guides/queries` indicates: - -- Library: `query` -- Path: `framework/react/guides/queries` - -### Deep Source Code Exploration - -For understanding library internals beyond what documentation provides (implementation details, type definitions, test patterns), clone the relevant repository directly: - -| Library | Clone Command | -| ------------ | --------------------------------------------------------- | -| Query | `git clone --depth 1 https://github.com/tanstack/query` | -| Router/Start | `git clone --depth 1 https://github.com/tanstack/router` | -| Table | `git clone --depth 1 https://github.com/tanstack/table` | -| Form | `git clone --depth 1 https://github.com/tanstack/form` | -| Virtual | `git clone --depth 1 https://github.com/tanstack/virtual` | -| Store | `git clone --depth 1 https://github.com/tanstack/store` | -| DB | `git clone --depth 1 https://github.com/tanstack/db` | - -Then use native file tools (grep, read) to explore implementations, types, and tests. - ---- - -## Rate Limits - -- **All operations**: 60 requests per minute diff --git a/docs/partner-placement.md b/docs/partner-placement.md new file mode 100644 index 000000000..0d7b35f94 --- /dev/null +++ b/docs/partner-placement.md @@ -0,0 +1,70 @@ +# Partner Placement + +Partner placement is split into three separate decisions: + +- **Eligibility:** whether a partner can appear for a surface, category, library, feature, or user context. +- **Tier:** the partner's commercial/display tier, which controls visual treatment and relative access to surfaces. +- **Order:** the sequence used inside a surface once eligible partners are known. + +The order policy should be explicit per surface. Avoid treating the partner array order or legacy score as the policy itself. + +## Order Strategies + +### `static-curated` + +Use for surfaces where the order is editorially or commercially curated and should remain stable. These surfaces can preserve the existing order, use partner seniority, or use private commercial priority when seniority is unavailable. + +Do not expose private commercial criteria in client-facing field names, comments, or API responses. If a private priority is required, keep the public/client value generic, such as `placementPriority` or an already-resolved ordered list. + +### `tier-rotated` + +Use for visible partner surfaces where partners in the same tier should cycle over time. Tier order remains fixed, but partners inside the same tier are selected with weighted-random ordering for each loaded app session and surface. + +The root loader creates the session seed, so SSR and hydration receive the same value through loader data. Client placement hooks read the root loader data directly and do not refresh it during normal navigation. Each surface combines its stable surface id with the session seed; avoid runtime-generated component ids in the seed because they can diverge across server and client rendering. Partners default to equal probability within their tier; a future `placementWeight` can bias same-tier odds without changing the ordering API. + +### `contextual-recommendation` + +Use when a surface behaves like a product recommendation or builder suggestion. Product fit should be established before commercial weighting affects order. This strategy currently preserves the legacy tier/priority behavior while giving recommendation surfaces a separate policy label. + +### `machine-readable` + +Use for AI- or crawler-facing outputs such as `llms.txt` and JSON feeds. These should be deterministic and relevance-first. Do not rotate them merely for logo fairness unless the downstream behavior has been designed and disclosed as such. + +## Reserved Rules + +Reserved rules should be narrow. For example, Cloudflare is reserved as the first deployment/hosting partner in any list that contains Cloudflare and other deployment partners. That should not imply Cloudflare is globally first across every partner surface; non-deployment partners can still appear before Cloudflare when the surface is mixed-category. + +Deployment action buttons are a deployment-only surface. Cloudflare remains first when present. Tier order is still preserved, and providers within the same tier can rotate by session. + +## Current Surfaces + +- Partner directory: `tier-rotated` for active partners, `static-curated` for previous partners. +- Partner grids and embeds: `tier-rotated`. +- Docs and blog rails: `tier-rotated`. +- Mobile docs strip: `tier-rotated`. +- Builder feature picker and starter partner suggestions: `contextual-recommendation`. +- Deploy action buttons: `tier-rotated` with deployment provider tiers preserved. +- `llms.txt` and `/api/data/partners`: `machine-readable`. + +## Analytics + +Partner impression and click events should include: + +- `partner_id` +- `placement` +- `slot_index` +- `partner_tier` +- `order_strategy` +- `rotation_seed` when the strategy rotates + +This lets reporting distinguish partner performance from placement policy and makes under/over-exposure easier to debug. + +## Contract Language + +Suggested external framing: + +> Partner tiers determine eligibility, visual treatment, reporting, and relative access to surfaces. Placement within the same tier may rotate or be curated depending on the surface. Some placements may include explicitly reserved rules for product, infrastructure, legal, or strategic reasons. + +For AI-assisted selection: + +> AI-assisted partner selection prioritizes user need and capability fit first. Partner tier may influence selection only among qualified options. diff --git a/docs/proposals/npm-watchlist-registry-draft.md b/docs/proposals/npm-watchlist-registry-draft.md new file mode 100644 index 000000000..1e6a4f612 --- /dev/null +++ b/docs/proposals/npm-watchlist-registry-draft.md @@ -0,0 +1,694 @@ +# Draft: NPM Watchlist Registry + +## Purpose + +Define a source-of-truth registry for tracked npm entities, rollups, and curated watchlists. + +This registry should replace the idea that `popular comparisons` are the canonical list. Popular comparisons can still exist, but they should derive from this registry. + +Rollups should be the reusable grouping abstraction. + +Watchlists should remain the primary user-facing saved and subscribed surface. Categories are metadata on entities, not the source of truth for ranking surfaces. + +## Design Goals + +- Model logical libraries, not just raw package names. +- Support legacy package rollups. +- Support reusable rollup definitions for ecosystems and grouped chart views. +- Keep curation explicit and reviewable in code. +- Make it easy to derive chart `packageGroups` for the existing UI. +- Leave room for future time-aware lineage rules without requiring them in v1. + +## Suggested File + +- `src/utils/npm-watchlists.ts` + +## Proposed Types + +```ts +export type NpmTrackedEntity = { + id: string + label: string + shortLabel?: string + description?: string + categories: Array< + | 'tanstack' + | 'data-fetching' + | 'routing' + | 'state' + | 'table' + | 'form' + | 'virtualization' + | 'testing' + | 'styling' + | 'build' + | 'validation' + | 'docs' + | 'framework' + | 'animation' + | 'tooling' + | 'database' + > + color?: string + packages: Array<{ + name: string + from?: string + to?: string + }> + lineageStrategy?: 'sum-all' | 'time-bounded' + benchmarkEligible?: boolean + popularComparisonEligible?: boolean + hidden?: boolean +} + +export type NpmWatchlist = { + id: string + title: string + description?: string + kind: 'curated-category' | 'curated-benchmark' | 'curated-tanstack' + entityIds?: string[] + rollupIds?: string[] + featured?: boolean + public?: boolean + popularComparison?: boolean + benchmark?: boolean +} + +export type NpmRollup = { + id: string + title: string + description?: string + kind: 'ecosystem' | 'category' | 'benchmark' | 'editorial' + entityIds: string[] + membershipMode?: 'exclusive' | 'overlap' + color?: string + public?: boolean +} +``` + +## V1 Simplification + +Even though the entity type allows `from` and `to`, v1 can treat almost all tracked entities as: + +- `lineageStrategy: 'sum-all'` +- all packages included for the full period + +That matches the current stats architecture and keeps the first implementation simple. + +## Derivations + +The registry should support these outputs: + +1. `getWatchlist(id)` +2. `getTrackedEntity(id)` +3. `getRollup(id)` +4. `getFeaturedWatchlists()` +5. `toPackageGroups(watchlist)` for existing chart routes +6. `toPopularComparisons()` to replace hand-maintained duplication +7. `groupEntitiesByRollup(entityIds, rollupIds)` for charting and digest summarization + +## Curation Rules + +### Tracked entities + +- One entity should represent one logical product or library line. +- One entity can belong to multiple categories. +- Legacy package names should be included when they clearly map to the same product lineage. +- Do not merge adjacent but meaningfully different products just because users might compare them. + +### Watchlists + +- A curated category watchlist should feel category-coherent. +- A benchmark watchlist should be broad, stable, and slow-changing. +- A featured watchlist should be editorially maintained and reviewed regularly. + +### Rollups + +- A rollup is a reusable editorial grouping of entities. +- Rollups are explicit, not inferred from npm scopes or package naming. +- Some rollups can overlap. +- Some rollups should be exclusive where double counting would distort rankings. +- Rollups should be stable enough to support long-term historical views. + +### Review cadence + +- Featured watchlists: monthly review +- Benchmark watchlists: quarterly review +- Ecosystem rollups: quarterly review +- Entity lineage changes: only when we have high confidence + +## Starter Tracked Entities + +This is not exhaustive. It is the first pass for v1. + +```ts +export const trackedEntities: NpmTrackedEntity[] = [ + { + id: 'tanstack-query', + label: 'TanStack Query', + categories: ['tanstack', 'data-fetching'], + color: '#FF4500', + packages: [{ name: '@tanstack/react-query' }, { name: 'react-query' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'swr', + label: 'SWR', + categories: ['data-fetching'], + color: '#ec4899', + packages: [{ name: 'swr' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'apollo-client', + label: 'Apollo Client', + categories: ['data-fetching'], + color: '#6B46C1', + packages: [{ name: '@apollo/client' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'trpc-client', + label: 'tRPC Client', + categories: ['data-fetching'], + color: '#2596BE', + packages: [{ name: '@trpc/client' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'tanstack-router', + label: 'TanStack Router', + categories: ['tanstack', 'routing'], + color: '#32CD32', + packages: [{ name: '@tanstack/react-router' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'react-router', + label: 'React Router', + categories: ['routing'], + color: '#FF0000', + packages: [{ name: 'react-router' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'wouter', + label: 'Wouter', + categories: ['routing'], + color: '#8b5cf6', + packages: [{ name: 'wouter' }], + popularComparisonEligible: true, + }, + { + id: 'tanstack-table', + label: 'TanStack Table', + categories: ['tanstack', 'table'], + color: '#FF7043', + packages: [{ name: '@tanstack/react-table' }, { name: 'react-table' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'ag-grid', + label: 'AG Grid', + categories: ['table'], + color: '#29B6F6', + packages: [{ name: 'ag-grid-community' }, { name: 'ag-grid-enterprise' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'mui-data-grid', + label: 'MUI Data Grid', + categories: ['table'], + color: '#1976D2', + packages: [{ name: '@mui/x-data-grid' }, { name: 'mui-datatables' }], + popularComparisonEligible: true, + }, + { + id: 'tanstack-form', + label: 'TanStack Form', + categories: ['tanstack', 'form'], + color: '#FFD700', + packages: [ + { name: '@tanstack/form-core' }, + { name: '@tanstack/react-form' }, + ], + popularComparisonEligible: true, + }, + { + id: 'react-hook-form', + label: 'React Hook Form', + categories: ['form'], + color: '#EC5990', + packages: [{ name: 'react-hook-form' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'conform', + label: 'Conform', + categories: ['form'], + color: '#FF5733', + packages: [{ name: '@conform-to/dom' }], + popularComparisonEligible: true, + }, + { + id: 'tanstack-virtual', + label: 'TanStack Virtual', + categories: ['tanstack', 'virtualization'], + color: '#8B5CF6', + packages: [{ name: '@tanstack/react-virtual' }, { name: 'react-virtual' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'react-window', + label: 'react-window', + categories: ['virtualization'], + color: '#4ECDC4', + packages: [{ name: 'react-window' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'react-virtualized', + label: 'react-virtualized', + categories: ['virtualization'], + color: '#FF6B6B', + packages: [{ name: 'react-virtualized' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'redux', + label: 'Redux', + categories: ['state'], + color: '#764ABC', + packages: [{ name: 'redux' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'zustand', + label: 'Zustand', + categories: ['state'], + color: '#764ABC', + packages: [{ name: 'zustand' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'jotai', + label: 'Jotai', + categories: ['state'], + color: '#6366f1', + packages: [{ name: 'jotai' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'valtio', + label: 'Valtio', + categories: ['state'], + color: '#FF6B6B', + packages: [{ name: 'valtio' }], + popularComparisonEligible: true, + }, + { + id: 'vite', + label: 'Vite', + categories: ['build', 'tooling'], + color: '#008000', + packages: [{ name: 'vite' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'webpack', + label: 'Webpack', + categories: ['build', 'tooling'], + color: '#8DD6F9', + packages: [{ name: 'webpack' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'rollup', + label: 'Rollup', + categories: ['build', 'tooling'], + color: '#e80A3F', + packages: [{ name: 'rollup' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'esbuild', + label: 'esbuild', + categories: ['build', 'tooling'], + color: '#FFCF00', + packages: [{ name: 'esbuild' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'rspack', + label: 'Rspack', + categories: ['build', 'tooling'], + color: '#8DD6F9', + packages: [{ name: '@rspack/core' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'zod', + label: 'Zod', + categories: ['validation'], + color: '#ef4444', + packages: [{ name: 'zod' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'valibot', + label: 'Valibot', + categories: ['validation'], + color: '#f97316', + packages: [{ name: 'valibot' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'yup', + label: 'Yup', + categories: ['validation'], + color: '#06b6d4', + packages: [{ name: 'yup' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'react', + label: 'React', + categories: ['framework'], + color: '#61DAFB', + packages: [{ name: 'react' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'vue', + label: 'Vue', + categories: ['framework'], + color: '#41B883', + packages: [{ name: 'vue' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'angular-core', + label: 'Angular', + categories: ['framework'], + color: '#DD0031', + packages: [{ name: '@angular/core' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'svelte', + label: 'Svelte', + categories: ['framework'], + color: '#FF3E00', + packages: [{ name: 'svelte' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, + { + id: 'solid-js', + label: 'Solid', + categories: ['framework'], + color: '#2C4F7C', + packages: [{ name: 'solid-js' }], + benchmarkEligible: true, + popularComparisonEligible: true, + }, +] +``` + +## Starter Curated Watchlists + +```ts +export const watchlists: NpmWatchlist[] = [ + { + id: 'data-fetching', + title: 'Data Fetching', + kind: 'curated-category', + entityIds: ['tanstack-query', 'swr', 'apollo-client', 'trpc-client'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'routing-react', + title: 'Routing (React)', + kind: 'curated-category', + entityIds: ['react-router', 'tanstack-router', 'wouter'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'tables-data-grids', + title: 'Tables and Data Grids', + kind: 'curated-category', + entityIds: ['ag-grid', 'tanstack-table', 'mui-data-grid'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'forms', + title: 'Forms', + kind: 'curated-category', + entityIds: ['react-hook-form', 'tanstack-form', 'conform'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'virtualization', + title: 'Virtualization', + kind: 'curated-category', + entityIds: ['react-virtualized', 'react-window', 'tanstack-virtual'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'state-management', + title: 'State Management', + kind: 'curated-category', + entityIds: ['redux', 'zustand', 'jotai', 'valtio'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'build-tools', + title: 'Build Tools', + kind: 'curated-category', + entityIds: ['webpack', 'vite', 'rollup', 'esbuild', 'rspack'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'validation', + title: 'Validation', + kind: 'curated-category', + entityIds: ['zod', 'valibot', 'yup'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'frameworks', + title: 'Frameworks', + kind: 'curated-category', + entityIds: ['react', 'vue', 'angular-core', 'svelte', 'solid-js'], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'all-tanstack', + title: 'All TanStack Libraries', + kind: 'curated-tanstack', + entityIds: [ + 'tanstack-query', + 'tanstack-router', + 'tanstack-table', + 'tanstack-form', + 'tanstack-virtual', + ], + featured: true, + public: true, + popularComparison: true, + }, + { + id: 'javascript-ecosystem-leaders', + title: 'JavaScript Ecosystem Leaders', + kind: 'curated-benchmark', + entityIds: [ + 'react', + 'vue', + 'angular-core', + 'svelte', + 'vite', + 'webpack', + 'rollup', + 'esbuild', + 'react-router', + 'tanstack-query', + 'apollo-client', + 'redux', + 'zustand', + 'react-hook-form', + 'zod', + ], + featured: true, + public: true, + benchmark: true, + }, +] +``` + +## Starter Rollups + +These are the new reusable grouping layer. + +```ts +export const rollups: NpmRollup[] = [ + { + id: 'tanstack-ecosystem', + title: 'TanStack Ecosystem', + kind: 'ecosystem', + entityIds: [ + 'tanstack-query', + 'tanstack-router', + 'tanstack-table', + 'tanstack-form', + 'tanstack-virtual', + ], + membershipMode: 'exclusive', + color: '#FF4500', + public: true, + }, + { + id: 'data-fetching-ecosystem', + title: 'Data Fetching Ecosystem', + kind: 'category', + entityIds: ['tanstack-query', 'swr', 'apollo-client', 'trpc-client'], + membershipMode: 'overlap', + public: true, + }, + { + id: 'router-ecosystem', + title: 'Router Ecosystem', + kind: 'category', + entityIds: ['react-router', 'tanstack-router', 'wouter'], + membershipMode: 'overlap', + public: true, + }, + { + id: 'javascript-ecosystem-index', + title: 'JavaScript Ecosystem Index', + kind: 'benchmark', + entityIds: [ + 'react', + 'vue', + 'angular-core', + 'svelte', + 'vite', + 'webpack', + 'rollup', + 'esbuild', + 'react-router', + 'tanstack-query', + 'redux', + 'react-hook-form', + 'zod', + ], + membershipMode: 'overlap', + public: true, + }, +] +``` + +Future ecosystem rollups can include: + +- `vercel-ecosystem` +- `remix-ecosystem` +- `shopify-ecosystem` + +Those should be explicit editorial definitions, not guessed from npm scopes. + +## Notes On Specific TanStack Entities + +### Query + +- Roll up `react-query` into `@tanstack/react-query`. +- This is a clear lineage case. + +### Table + +- Roll up `react-table` into `@tanstack/react-table`. +- This is also a clear lineage case. + +### Virtual + +- Roll up `react-virtual` into `@tanstack/react-virtual`. +- Clear lineage case. + +### Router + +- Do not automatically roll `react-location` into `@tanstack/react-router` in v1. +- They are adjacent and related, but this one is more semantically debatable than Query, Table, or Virtual. +- Keep this as an explicit later decision if we want lineage continuity there. + +### Form + +- It may be useful to define separate tracked entities for `@tanstack/form-core` and `@tanstack/react-form` later. +- For now, a single `TanStack Form` entity is probably fine if the goal is product-level visibility. + +## Migration From Existing Popular Comparisons + +Recommended path: + +1. Create the entity, rollup, and watchlist registry. +2. Rebuild `getPopularComparisons()` from curated watchlists and rollups marked for that purpose. +3. Keep route behavior and `packageGroup` format unchanged at first. +4. Add optional rollup grouping to chart UI later. +5. Remove duplication once parity looks good. + +## Open Decisions + +1. How aggressive should we be about rolling up framework adapters into one product-level TanStack entity? +2. Which rollups should be exclusive versus overlap-friendly? +3. Should benchmark lists include only `benchmarkEligible` entities or also allow manual exceptions? +4. Should `JavaScript Ecosystem Leaders` aim for 25, 50, or 100 entities in v1? +5. Which watchlists and rollups should be exposed publicly on day one versus internal-only at first? + +## Immediate Next Steps + +- Convert this draft into a real `src/utils/npm-watchlists.ts` module. +- Add real rollup definitions for ecosystem and benchmark views. +- Fill out the tracked entity registry beyond the starter set. +- Rebuild stale popular comparisons from the registry. +- Draft the first larger benchmark roster separately. diff --git a/docs/proposals/npm-watchlists-and-weekly-digests.md b/docs/proposals/npm-watchlists-and-weekly-digests.md new file mode 100644 index 000000000..9abc6e304 --- /dev/null +++ b/docs/proposals/npm-watchlists-and-weekly-digests.md @@ -0,0 +1,665 @@ +# Proposal: NPM Watchlists and Weekly Digests + +## North Star + +Turn TanStack NPM Stats into a durable market map for JavaScript libraries, not just a charting tool. + +Users should be able to follow a set of libraries, understand who is actually gaining or losing ground inside that set, and get a concise weekly digest that surfaces meaningful movement without reacting to noisy daily swings. + +## Why This Matters + +- Raw npm download charts are useful, but they do not answer "who is winning?" +- Raw download growth is increasingly misleading because the whole ecosystem keeps growing. +- Users care about cohorts, rankings, trend changes, and share shifts more than isolated download counts. +- TanStack already has fast comparison UX and historical npm data. This feature turns that into a repeatable product. + +## Product Outcome + +Add rollups and watchlists that users can explore, compare, and subscribe to by email once a week. + +Each digest should answer: + +- Which libraries moved up or down? +- Which libraries gained or lost share? +- Which crossovers happened? +- Which trends look sustained instead of noisy? + +## Core Product Principles + +- `packages -> entities -> rollups -> watchlists` should be the core model. +- Watchlists are the user-facing saved and subscribed surface. +- Rollups are reusable editorial groupings for charts, rankings, and ecosystem views. +- Popular comparisons are just one derived view, not the source of truth. +- Rankings should be cohort-aware, not framed as vague "global npm rank". +- Normalization should favor share and relative outperformance, not raw download growth. +- Long-term rollups must work across legacy package names and package renames. +- Weekly summaries should highlight meaningful movement, not short-term volatility. + +## User Stories + +1. As a user, I can subscribe to a curated watchlist like `Data Fetching` or `Build Tools`. +2. As a user, I can save a custom watchlist from the current `/stats/npm` comparison. +3. As a user, I can follow a logical library even if its npm history spans multiple package names. +4. As a user, I can receive a weekly digest showing rank changes, share changes, and notable crossovers. +5. As a user, I can look back across months or years and see how the watchlist evolved. +6. As a user, I can compare larger ecosystem rollups like `TanStack` vs `Vercel` vs `Remix`. +7. As a user, I can plot entities grouped by rollup on charts. + +## Product Shape + +### 1. Tracked entities + +The atomic unit should not be a single npm package. It should be a logical tracked entity. + +Example: + +```ts +{ + id: 'tanstack-query', + label: 'TanStack Query', + packages: ['@tanstack/react-query', 'react-query'], +} +``` + +This gives us: + +- continuity across legacy/current package names +- stable ranking entities +- cleaner digest language +- long-term rollups without losing historical identity + +### 2. Rollups + +A rollup is a reusable grouped view of tracked entities. + +Examples: + +- `tanstack-ecosystem` +- `vercel-ecosystem` +- `remix-ecosystem` +- `data-fetching-ecosystem` +- `router-ecosystem` + +Rollups let us: + +- compare larger ecosystems on charts +- group entities visually inside one watchlist +- create reusable market-map views without copying entity membership everywhere +- support ecosystem-vs-ecosystem reporting in digests later + +### 3. Watchlists + +A watchlist is a named saved surface for ranking, digesting, and charting. + +A watchlist can contain: + +- entities directly +- rollups +- or both, depending on the UX we want + +Recommended watchlist types: + +- Curated category watchlists +- Curated benchmark watchlists +- TanStack-specific watchlists +- Ecosystem watchlists +- User-created custom watchlists + +### 4. Weekly digest + +Each watchlist can produce a weekly digest with: + +- current rank +- previous rank +- rank delta +- share of watchlist +- share delta +- trailing 4-week trend +- notable crossovers +- link back to the live chart + +Future digest expansions: + +- rollup-vs-rollup share changes +- entity movement inside a rollup +- ecosystem momentum summaries + +## Current Model Draft + +This is the current preferred abstraction stack: + +- `package`: raw npm package history +- `entity`: one logical library or product across one or more packages +- `rollup`: reusable editorial grouping of entities +- `watchlist`: a saved and subscribable surface for charts, rankings, and digests + +Current preferred type direction: + +```ts +export type NpmTrackedEntity = { + id: string + label: string + shortLabel?: string + description?: string + categories: Array< + | 'tanstack' + | 'data-fetching' + | 'routing' + | 'state' + | 'table' + | 'form' + | 'virtualization' + | 'testing' + | 'styling' + | 'build' + | 'validation' + | 'docs' + | 'framework' + | 'animation' + | 'tooling' + | 'database' + > + color?: string + packages: Array<{ + name: string + from?: string + to?: string + }> + lineageStrategy?: 'sum-all' | 'time-bounded' + benchmarkEligible?: boolean + popularComparisonEligible?: boolean + hidden?: boolean +} + +export type NpmRollup = { + id: string + title: string + description?: string + kind: 'ecosystem' | 'category' | 'benchmark' | 'editorial' + entityIds: string[] + membershipMode?: 'exclusive' | 'overlap' + color?: string + public?: boolean +} + +export type NpmWatchlist = { + id: string + title: string + description?: string + kind: 'curated-category' | 'curated-benchmark' | 'curated-tanstack' + entityIds?: string[] + rollupIds?: string[] + featured?: boolean + public?: boolean + popularComparison?: boolean + benchmark?: boolean +} +``` + +Important modeling notes: + +- categories belong on entities as `categories: string[]`, not a single `category` +- watchlists are still the primary curated product surface +- rollups are the reusable grouping abstraction +- popular comparisons should be derived from the registry, not hand-maintained independently +- v1 can mostly use `lineageStrategy: 'sum-all'` +- the model should still leave room for future time-bounded lineage + +## Current Decisions + +These are the main decisions made so far and should be preserved in any handoff. + +- Do not frame the feature as "global npm rank" in v1. +- Rank within a defined cohort, watchlist, or benchmark basket. +- Use trailing 28-day downloads for ranking. +- Use share of watchlist as the primary normalization lens. +- Use relative growth vs watchlist median as a secondary lens. +- Defer anomaly detection. +- Prefer conservative digest-oriented trend signals over loose anomaly alerts. +- Build curated entities and watchlists separately from current `popular comparisons`. +- Include legacy package rollups where lineage is clear. +- Preserve compatibility with long-term and multi-year rollups. +- Treat rollups as first-class because ecosystem-vs-ecosystem views are valuable. +- Allow entities to belong to multiple categories. +- Allow some rollups to overlap and some to be exclusive. + +## Lineage Decisions So Far + +These are the explicit lineage calls already discussed. + +- Roll up `react-query` into `@tanstack/react-query`. +- Roll up `react-table` into `@tanstack/react-table`. +- Roll up `react-virtual` into `@tanstack/react-virtual`. +- Do not automatically roll `react-location` into `@tanstack/react-router` in v1. +- TanStack Form may eventually need separate entity treatment for `@tanstack/form-core` and `@tanstack/react-form`, but one product-level entity is acceptable for v1. + +## Starter Registry Direction + +These are the concrete examples already drafted and should be treated as the current starting point, not final production data. + +Starter entities: + +- `tanstack-query`: `@tanstack/react-query` + `react-query` +- `swr`: `swr` +- `apollo-client`: `@apollo/client` +- `trpc-client`: `@trpc/client` +- `tanstack-router`: `@tanstack/react-router` +- `react-router`: `react-router` +- `wouter`: `wouter` +- `tanstack-table`: `@tanstack/react-table` + `react-table` +- `ag-grid`: `ag-grid-community` + `ag-grid-enterprise` +- `mui-data-grid`: `@mui/x-data-grid` + `mui-datatables` +- `tanstack-form`: `@tanstack/form-core` + `@tanstack/react-form` +- `react-hook-form`: `react-hook-form` +- `conform`: `@conform-to/dom` +- `tanstack-virtual`: `@tanstack/react-virtual` + `react-virtual` +- `react-window`: `react-window` +- `react-virtualized`: `react-virtualized` +- `redux`: `redux` +- `zustand`: `zustand` +- `jotai`: `jotai` +- `valtio`: `valtio` +- `vite`: `vite` +- `webpack`: `webpack` +- `rollup`: `rollup` +- `esbuild`: `esbuild` +- `rspack`: `@rspack/core` +- `zod`: `zod` +- `valibot`: `valibot` +- `yup`: `yup` +- `react`: `react` +- `vue`: `vue` +- `angular-core`: `@angular/core` +- `svelte`: `svelte` +- `solid-js`: `solid-js` + +Starter watchlists: + +- `data-fetching` +- `routing-react` +- `tables-data-grids` +- `forms` +- `virtualization` +- `state-management` +- `build-tools` +- `validation` +- `frameworks` +- `all-tanstack` +- `javascript-ecosystem-leaders` + +Starter rollups: + +- `tanstack-ecosystem` +- `data-fetching-ecosystem` +- `router-ecosystem` +- `javascript-ecosystem-index` + +Future rollups already discussed: + +- `vercel-ecosystem` +- `remix-ecosystem` +- `shopify-ecosystem` + +## Rollup Semantics + +Rollups should be explicit editorial definitions, not inferred automatically from npm scopes or package naming. + +Two intended rollup modes: + +- `exclusive`: best for aggregate comparisons where double counting would distort the view +- `overlap`: best for discovery, category views, and flexible grouped charting + +Examples: + +- `tanstack-ecosystem` should likely be `exclusive` +- category-style rollups like `router-ecosystem` can be `overlap` + +## Popular Comparisons Migration + +The current `popular comparisons` file is useful but not a good source of truth. + +Migration direction: + +1. Build the entity, rollup, and watchlist registry. +2. Rebuild `getPopularComparisons()` from curated watchlists and rollups flagged for chart use. +3. Keep the existing route behavior and `packageGroup` output shape unchanged at first. +4. Add rollup-aware chart grouping later. +5. Remove duplicated manual comparison definitions once parity looks good. + +## Handoff Notes + +If another agent picks this up, these are the highest-value next steps already implied by the current plan. + +1. Convert the draft model into a real `src/utils/npm-watchlists.ts` module. +2. Expand the entity registry beyond the starter examples. +3. Define the first real ecosystem rollups. +4. Decide the first public benchmark basket, especially `JavaScript Ecosystem Leaders`. +5. Replace stale `popular comparisons` with registry-derived definitions. +6. Design the DB schema around entities, rollups, watchlists, subscriptions, and weekly snapshots. + +## Registry Strategy + +### Entities, rollups, and watchlists should become a first-class registry + +Current `popular comparisons` are useful, but they are not sufficient as the long-term source of truth. + +Problems with using them directly: + +- some are out of date +- some are sized for chart exploration, not ongoing tracking +- some do not include legacy package rollups +- some are too small or too editorial for ranking semantics + +We should create a separate curated registry for entities, rollups, and watchlists and let `popular comparisons` derive from it where useful. + +Suggested source file: + +- `src/utils/npm-watchlists.ts` + +### Why rollups matter + +Rollups are what let this feature graduate from simple package comparison into a real market map. + +Examples: + +- compare `TanStack` vs `Vercel` vs `Remix` +- plot all router-related entities and color them by ecosystem rollup +- compare a product rollup against its category peers +- track how an ecosystem's aggregate share changes over time + +### Starter watchlist categories + +Curated category watchlists: + +- Data Fetching +- Routing +- Tables and Data Grids +- Forms +- State Management +- Virtualization +- Testing +- Styling +- Build Tools +- Validation and Schema +- Motion and Animation +- Meta-frameworks + +Curated benchmark watchlists: + +- JavaScript Ecosystem Leaders +- React Ecosystem Leaders +- JavaScript Infra Index +- Frontend Foundation 50 + +TanStack watchlists: + +- All TanStack Libraries +- TanStack Query Ecosystem +- TanStack Router Ecosystem +- TanStack Table Ecosystem + +Ecosystem rollups and watchlists: + +- TanStack Ecosystem +- Vercel Ecosystem +- Remix Ecosystem +- Router Ecosystem +- Data Fetching Ecosystem + +## Legacy Package Rules + +Legacy packages should usually roll into the same tracked entity when they represent the same product lineage. + +Examples: + +- `react-query` + `@tanstack/react-query` +- `react-table` + `@tanstack/react-table` +- `react-virtual` + `@tanstack/react-virtual` + +Do not assume every rename or adjacent product belongs in the same lineage forever. The model should allow future time-aware lineage rules if simple summing becomes misleading. + +V1 rule: + +- tracked entities use a flat array of package names and simple summed downloads + +Future rule if needed: + +- tracked entities can support time-bounded package membership + +## Ranking and Normalization + +Raw download growth is not a good primary signal because the entire ecosystem is growing. + +### Primary ranking metric + +Use trailing 28-day downloads for rank within a watchlist. + +Why 28-day: + +- smoother than daily or weekly +- still responsive enough for weekly digests +- less sensitive to npm reporting noise + +### Primary normalization metric: share of watchlist + +```txt +share_of_watchlist = entity_28d_downloads / sum(all_watchlist_entity_28d_downloads) +``` + +This should be the main lens in digests because it makes leaders and laggards visible even when the whole cohort is growing. + +### Secondary normalization metric: relative growth vs cohort + +```txt +relative_growth = entity_growth_rate - median_growth_rate_of_watchlist +``` + +This helps answer whether a library is outperforming or underperforming peers. + +### Optional tertiary benchmark: ecosystem index + +Do not use a raw sum of giant packages as the main baseline. + +If we add an ecosystem benchmark, it should come from a stable curated basket like `JavaScript Ecosystem Leaders` and use a median or trimmed-mean growth factor rather than raw weighted sum. + +### Digest metrics to show in v1 + +- rank +- rank delta +- share of watchlist +- share delta +- trailing 4-week trend +- crossover events + +## Trend Detection + +Avoid generic "anomaly detection" in v1. + +The data is too noisy and npm has known reporting weirdness. We already correct obvious outliers and zero-day anomalies in the stats pipeline. + +If we surface trend change signals, they should be conservative and digest-oriented. + +Suggested v1 notable movement rules: + +- rank changed by at least 1 +- share moved by a meaningful threshold +- one entity crossed another and stayed ahead for at least one full digest period +- growth outperformed the cohort median by a meaningful margin + +Possible later signals: + +- sustained acceleration over 4 weeks vs prior 8 to 12 weeks +- sustained deceleration over 4 weeks vs prior 8 to 12 weeks +- held rank breakout after 2 consecutive weeks + +## Long-Term Rollups + +This feature must support multi-year rollups. + +The current architecture is already favorable: + +- npm history is cached as year-based daily chunks +- historical chunks are immutable +- package groups already support rollups across multiple packages + +What we need to preserve: + +- tracked entities, not single-package rows +- reusable rollups on top of tracked entities +- raw historical chunks as source of truth +- derived weekly or monthly snapshots for cheap leaderboard queries + +Recommended derived data: + +- weekly watchlist snapshots for digest generation and rank history +- optional monthly snapshots for faster long-range reporting + +This keeps the system flexible: + +- raw chunks allow recomputation when formulas change +- snapshots keep digests and rank-history queries cheap +- rollups can be recalculated without changing the raw source data + +### Rollup views to support later + +- yearly rollup trend lines +- rank history by entity within a rollup +- ecosystem-vs-ecosystem comparisons +- grouped charts where entities are colored or faceted by rollup + +## UX Entry Points + +### `/stats/npm` + +- `Save this comparison` +- `Follow this watchlist` +- `Subscribe to weekly digest` + +### Library-specific npm stats pages + +- `Follow this library` +- `Add to watchlist` + +### Account area + +- My watchlists +- Subscriptions +- Digest frequency +- Pause or unsubscribe + +## Data Model Direction + +Suggested tables: + +- `npm_tracked_entities` +- `npm_tracked_entity_packages` +- `npm_rollups` +- `npm_rollup_entities` +- `npm_watchlists` +- `npm_watchlist_items` +- `npm_watchlist_subscriptions` +- `npm_watchlist_weekly_snapshots` +- `npm_digest_sends` + +Notes: + +- tracked entities are the canonical source for rollups and labels +- rollups reference tracked entities +- watchlists can reference tracked entities, rollups, or both +- weekly snapshots are required for stable digest generation and historical ranking views +- digest send records help with idempotency and auditability + +## Implementation Phases + +### Phase 1. Curated registry + +- Define tracked entities, rollups, and curated watchlists in code +- Include legacy package rollups where appropriate +- Refresh and replace stale `popular comparisons` +- Establish one larger benchmark watchlist like `JavaScript Ecosystem Leaders` + +### Phase 2. Snapshot pipeline + +- Compute weekly watchlist snapshots from historical npm data +- Store rank, share, and trailing trend fields +- Make snapshot generation idempotent + +### Phase 3. User-facing watchlists + +- Allow signed-in users to save comparisons as custom watchlists +- Allow subscription management from account surfaces +- Allow subscription from stats pages + +### Phase 4. Weekly digests + +- Generate digest content from weekly snapshots +- Send through Resend +- Include unsubscribe and pause controls + +### Phase 5. Trend signals and benchmark refinement + +- Add conservative trend-change detection +- Add ecosystem index if the curated benchmark proves stable and useful + +## Suggested Implementation Order + +1. Build the curated tracked-entity and watchlist registry. +2. Add reusable rollups for ecosystems and category-level groupings. +3. Replace or derive `popular comparisons` from the new registry. +4. Add database tables for tracked entities, rollups, watchlists, subscriptions, and weekly snapshots. +5. Build the weekly snapshot job on top of existing historical npm chunks. +6. Expose watchlists and rollup grouping in UI without email first. +7. Add weekly digest generation and delivery. +8. Add rank history and long-range watchlist views. +9. Add conservative trend signals only after real snapshot data exists. + +## Success Criteria + +- Users can subscribe to curated watchlists and custom watchlists. +- Users can compare major ecosystem rollups clearly. +- Digests clearly show rank movement and share movement. +- Rankings are stable enough to feel trustworthy week to week. +- Legacy package history rolls up cleanly into one logical entity. +- Multi-year watchlist history remains queryable and understandable. +- We can explain the ranking math in plain English. + +## Non-Goals For V1 + +- claiming true global npm rank across the whole registry +- real-time alerts +- daily digest spam +- loose anomaly detection with low confidence +- overfitting the system to every one-off package rename + +## Open Questions + +1. Which curated watchlists should ship first? +2. How large should `JavaScript Ecosystem Leaders` be? +3. Which legacy package transitions deserve permanent lineage rollups? +4. Which rollups should be exclusive versus allowed to overlap? +5. Should custom watchlists allow raw package groups, tracked entities, rollups, or all three? +6. Should digests be weekly only in v1, or should monthly be supported from day one? +7. Do we want one digest per watchlist or one combined digest across all subscriptions? + +## Progress + +- [ ] Define tracked entity model +- [ ] Define rollup model +- [ ] Draft curated watchlist registry +- [ ] Draft curated ecosystem rollups +- [ ] Audit and refresh current popular comparisons +- [ ] Define lineage rules for legacy package rollups +- [ ] Design DB schema +- [ ] Build weekly snapshot job +- [ ] Build watchlist UI surfaces +- [ ] Build subscription management +- [ ] Build weekly digest generation +- [ ] Build email delivery and unsubscribe flow +- [ ] Add rank history views +- [ ] Evaluate trend signals after snapshot data accumulates + +## Notes + +- Existing npm stats infrastructure already gives us a strong foundation: historical daily chunks, package grouping, outlier correction, and email transport. +- The biggest risk is product semantics, not raw implementation difficulty. +- The feature will feel credible only if watchlists are curated well and ranking math stays easy to explain. diff --git a/docs/source-code-audit-2026-06-23.md b/docs/source-code-audit-2026-06-23.md new file mode 100644 index 000000000..cd5da7ed0 --- /dev/null +++ b/docs/source-code-audit-2026-06-23.md @@ -0,0 +1,2392 @@ +# Source Code Audit - 2026-06-23 + +Status: living audit report. Scope covers the source inventory, static scans, full `src/routes` pass, `src/libraries` metadata, `src/styles`, API/server-function boundaries, auth/OAuth repositories and flows, builder endpoints, builder client/deploy UI, application-starter paths, `src/components` domain passes including landing, shop, game, stats, docs, admin/community, SearchModal/AiDock, LibraryLayout/docs chrome, partner/sponsor, stack, CodeExplorer/FileExplorer, query/hooks/client data helpers, MCP transport/tools, CLI auth tickets, scheduled tasks, shared cache/db/runtime/Sentry/env helpers, UI primitives, image helpers, contexts/stores, Vite/Start/test config, scripts, and existing tests. Excludes `node_modules`, generated route tree findings, blog prose findings, images/assets, and packaged starter template files. + +The repo is large: roughly 145k hand-written TS/TSX lines under `src`, plus tests and scripts, with the largest files concentrated in search, navigation/layout, stats, builder, docs, admin/community flows, partners, and the game. + +## Merged Fix Order + +This is ordered by easy or quick work that also makes later fixes safer. Severity still overrides the queue when needed, so a P1 can jump the line, but this is the default sequence for batching. + +Tracking model: work this list top to bottom. Leave untouched bullets plain, prefix the active one with `[doing]`, completed ones with `[done YYYY-MM-DD: verification]`, and skipped/deferred ones with `[defer: reason]`. Add one short entry to the batch log when a batch lands. + +### 1. Small shared guardrails + +These are quick, high-leverage patches because later fixes can reuse them instead of adding another local workaround. + +- [done 2026-06-24: `pnpm run test:unit`, `pnpm test`] Make `pnpm test` honest about TypeScript tests, or split script names so validation expectations are clear. +- [done 2026-06-24: `pnpm run test:tsc`, `pnpm run test:lint`] Default non-submit buttons/select triggers to `type="button"`, fix the shared pagination label/id contract, and clean up tooltip child prop merging. +- [done 2026-06-24: `pnpm run test:unit`, `pnpm run test:tsc`, `pnpm run test:lint`] Add tiny helpers for response filenames, package slug encoding, and row-local ids. +- [done 2026-06-24: `pnpm test`, 60-route local smoke] Add tiny helpers for external URL normalization, internal route classification, and guarded storage. +- [done 2026-06-24: `pnpm test`, 60-route local smoke] Share low-risk clipboard/copy-state timers and auth popup centering. +- Fix custom `useMutation` stale callbacks, docs sidebar timers, remaining popup blocked-state handling, and remaining page-visibility/reduced-motion call sites. +- [done 2026-06-24: `pnpm test`, 60-route local smoke] Normalize maintainer bare-domain URLs, add Scarf/social URL contract tests, and centralize the public-library selector. +- Normalize partner analytics seed buckets, blog image URLs, and host cache purge response policy. + +### 2. One-off cleanup that lowers noise + +Do these early when they're obvious and isolated, but don't let them block higher-leverage guardrails. + +- Fix Twoslash invalid CSS, duplicate shop fonts, shop `NEW` badge duration, app-starter dead selection parameter, app-starter schema caveat, Netlify pending state, mobile merch loading, sponsor tooltip math, Stack category fallbacks, and unsupported admin timestamp formatting. +- Remove or quarantine dead code once confirmed: old logo copies, unused account feedback component, showcase helper duplicates, unused shop promo components, deprecated `reactChartsProject`, and the old R3F scene/dependencies if we're ready to delete them. +- Lazy-load obvious bundle leaks with low behavior risk: Start/Router landing prompt resolver and navbar merch product loading. + +### 3. Fast security and public-boundary hardening + +These are small enough to do before deeper refactors and they reduce risk while larger helpers are being designed. + +- [done 2026-06-24: `pnpm test:tsc`, `pnpm test:lint`, `pnpm test`] Analytics proxy header allowlist, GitHub webhook fail-closed secret, Cloudflare-first IP extraction, route response filename sanitizing, Discord raw-body size/signature-shape guard, MCP transport content-type/length guard and masked errors. +- [done 2026-06-24: `pnpm test:tsc`, `pnpm test:lint`, `pnpm test`] Public pagination and route-search caps, admin pagination/range caps, docs feedback content caps, showcase field/array caps, Shopify page-size/quantity/discount caps, UploadThing client preflight caps, image transform clamps, community-resource frontmatter schema. +- [done 2026-06-24: `pnpm test:tsc`, `pnpm test:lint`, `pnpm test`] URL/link hardening: Intent metadata URLs, Intent tarball source paths, markdown/source/search link classification, legacy redirect segment matching, local docs path containment. +- [done 2026-06-24: `pnpm test:tsc`, `pnpm test:lint`, `pnpm test`] OAuth/login quick fixes: return-to key mismatch, login modal stale callbacks, provider route param picklists, session revocation conflict retry, and session base64 fallback removal once covered. + +### 4. Small controllers that prevent repeat bugs + +These are still bounded, but they should come before domain cleanup so every domain fix uses the same shape. + +- Route-scoped pagination controller with request fingerprints for shop, search, admin lists, stats, and any accumulated-page UI. +- Query-key completeness helper or lint pattern for related showcases, current-user data, npm recent downloads, GitHub stats, and cache field preservation. +- Optimistic mutation controller for showcase voting, feedback updates, keyed row actions, and cart/update flows. +- Browser effect toolkit for storage, timers, outside-click, drag/resize, clipboard, popup polling, window-open, and page visibility. +- Product option/color helper for shop cards, drawers, product pages, swatches, variant resolution, and preload budgets. + +### 5. Domain hardening with shared primitives in place + +At this point the repeatable helpers exist, so these PRs should be smaller and more reliable than fixing every route in isolation. + +- Stats: public npm fan-out caps, stats UI bounds, partial recent-cache semantics, baseline math, scheduled 429 backoff/deadlines, admin cache budgets, chart range caps, and d3/date-helper split. +- Builder/application starter: selected-framework validation, URL/store feature parsing, custom template/add-on contract, ecommerce template preservation, response parsing, transient feedback, nested feature-card links, and API endpoint guard alignment. +- Showcase/feedback: concurrent first-vote handling, query keys, submission/rank caps, moderation notification handling, detail/admin error states, audit target ids, leaderboard SQL aggregation, docs feedback cache/update helper. +- Shop: Storefront caps, trusted HTML policy, image transform helper, load-more fingerprints, variant option resolution, quick-view modal semantics, nested product-card interactions, preload budgets, cart optimistic shape, and mutation-backed success state. +- Admin: capability validators, all-table role/user queries, stale bulk selections, keyed row actions, invalidation scope, dashboard lazy tabs, route/server guard sync, and transaction boundaries where the write surface is small. +- Docs/markdown: frontmatter resilience, raw HTML/iframe trust policy, tab state parsing/recovery, remote docs config caps, recursive-tree truncation handling, and manifest/cache budgets. +- Game: progression/restored counts, compass/AI island sets, NaN guards, health flash, persistence ownership, async disposal checks, culling children, and geometry ownership. + +### 6. Larger boundary refactors + +These need design notes first, but the earlier guardrails should make the migration less risky. + +- One builder API request/response contract across compile, validate, download, deploy, remote loads, feature artifacts, suggestions, MCP, and client generation. +- One outbound fetch policy with timeout, status, content-type, response-size, redirect, and header rules for OAuth, GitHub, docs, Tranco, stats, Intent, Shopify, remote builder loads, and scripts. +- Intent ingestion redesign: remove public-read ingestion, add tarball budgets, failed-version backoff/dead-letter state, transactional skill replacement, admin process/list caps, complete GitHub discovery, and safe source paths. +- Auth/OAuth storage model: atomic authorization-code consumption, dynamic client registration persistence, client-id to redirect-uri binding, OAuth user/account upsert transaction, plaintext token migration. +- Docs manifest/cache system: recursive tree completeness, file-count/concurrency budgets, redirect metadata caching, stale fallback completeness, local path guards, and cache admin budgets. +- MCP and CLI auth: durable CLI tickets, public-create limits, rate-limit uniqueness by identifier type, cleanup by window size, API-key lifecycle caps, and lower `lastUsedAt` write amplification. +- Search/AI and LibraryLayout: split expensive imports, dock/modal state, Kapa/chat persistence, source-link routing, docs nav/tabs/mobile/sponsors/feedback, and route data boundaries. +- Partner/sponsor/catalog and shop storefront boundaries: split serializable metadata from assets/JSX, centralize public catalog visibility, remove sponsor duplication, and unify shop product/cart contracts. +- Environment/observability/runtime: consolidate env readers, Sentry PII/sample-rate policy, production diagnostics behavior, database context proxy, and script runtime budgets. + +### 7. Product-wide initiatives and upstream candidates + +Treat these as tracked initiatives, not normal cleanup PRs. + +- Product landing shell and library landing route factory. +- UI primitive consolidation across root and shop. +- Route/module boundary cleanup and shared validation schema system. +- Admin table/filter framework and docs feedback DOM primitive. +- Deploy-dialog controller and browser effect toolkit. +- Bundle-boundary program with reachability checks, global-shell lazy panels, partner asset split, AI/search split, d3/Plot splits, navbar merch lazy import, and old R3F dependency removal. +- Upstream candidates: TanStack Start API-boundary helper, outbound fetch helper, OAuth PKCE/code-consumption helper, Drizzle transaction/audit helper, route search hydration helper, endpoint request/response schema helper, package slug codec, docs manifest builder, npm download chunk cache, Shopify image helper. +- First AI skills to actually build: API hardening, outbound fetch hygiene, route param boundaries, query-key completeness, type-safety sweep, bundle hotspot splitting, UI primitive form defaults, and route reachability/dead-stack cleanup. + +## Batch Log + +- 2026-06-24: Landed the grouped security pass across public request boundaries, OAuth/login/session flows, builder/GitHub deploy APIs, outbound fetches, stats/Intent/docs/shop/showcase caps, UploadThing/image preflight, markdown/search link classification, docs redirects/path containment, and community-resource frontmatter validation. +- 2026-06-24: Second pass simplified the security patch by sharing bounded body readers, npm package-name/page/date/chart schemas, and URL normalization; removed the local application-starter request guard while preserving same-origin/content-type/body-size enforcement. +- 2026-06-24: Added shared response filename/content-disposition helpers, package route slug encoding helpers, and row-local id helpers; wired them into builder/docs downloads, Intent registry links, and moderation note inputs. +- 2026-06-24: Hardened shared UI primitives: defaulted shop buttons and shared select triggers to non-submit buttons, gave pagination a unique page-size label/id pair plus non-submit controls, and made Tooltip merge trigger handlers/refs without `any`. +- 2026-06-24: Added `test:unit` and wired it into `pnpm test` so existing TypeScript assertion tests run in the default validation path. +- 2026-06-24: Added guarded browser storage/effect helpers, migrated low-risk clipboard/timer/popup/storage/reduced-motion call sites, centralized public-library selection, normalized maintainer URLs, and added static data contract tests. +- 2026-06-23: Audit created and ordered. + +## Highest Priority Findings + +### P0/P1 - Analytics proxy forwards private request headers + +`src/server.ts:85-112` proxies same-origin analytics paths to Google and passes `headers: request.headers` directly at `src/server.ts:101-104`. + +Same-origin browser requests can include cookies such as session cookies and may include authorization-like headers. Forwarding the whole inbound header bag to Google is unnecessary and risky. + +Suggested fix: + +- Build a fresh `Headers` allowlist for analytics upstream requests. +- Never forward `cookie`, `authorization`, `host`, `cf-*`, `x-forwarded-*`, or internal headers. +- Consider allowing only `accept`, `accept-language`, `user-agent`, and content headers needed by collect POSTs. + +### P1 - OAuth authorization codes are not consumed atomically + +`src/auth/oauthClient.server.ts:168-238` reads an authorization code, validates expiry/redirect/PKCE, deletes it at `src/auth/oauthClient.server.ts:205-208`, then inserts access and refresh tokens. + +Two concurrent exchanges can read and validate the same code before either delete wins. Both can mint tokens. The schema has a unique `codeHash`, but that only prevents duplicate codes, not concurrent reuse. + +Suggested fix: + +- Use a transaction and atomically delete/claim by `codeHash` plus validity conditions, returning the row. +- Alternatively add a `consumedAt` field and update `where consumedAt is null returning`. +- Only mint tokens after a single request has proven it owns the code. + +### P1 - OAuth client IDs are not bound to registered redirect URIs + +`src/routes/oauth/register.ts:28-80` accepts dynamic client metadata, validates submitted redirect URIs, then returns a deterministic client id from `client_name` at `src/routes/oauth/register.ts:113-127`. The registration is not stored. Later, `src/routes/oauth/authorize.tsx:38-93` and `src/utils/oauthClient.functions.ts:48-79` only validate that the requested redirect URI is localhost or HTTPS; they do not verify that the `client_id` owns that redirect URI. + +The authorize UI displays `Authorize {displayClientId}` at `src/routes/oauth/authorize.tsx:241-254` but does not show the redirect origin. A crafted authorization link can therefore use a familiar-looking client id with a different HTTPS redirect URI. + +Suggested fix: + +- Store dynamic client registrations with redirect URI allowlists, client display names, and created timestamps. +- Require authorize/token flows to match `client_id + redirect_uri` against the stored registration. +- Add client-name, redirect-uri count, redirect-uri length, state, scope, and PKCE length caps. +- Show the redirect origin/app name in the consent screen. + +### P1 - Client IP extraction trusts spoofable headers before Cloudflare + +`src/utils/request.server.ts:21-34` checks `x-forwarded-for`, then `x-real-ip`, then `cf-connecting-ip`. + +On Cloudflare-hosted traffic, `cf-connecting-ip` should be the trusted source. Taking `x-forwarded-for` first can let clients spoof IP identity for rate limits, audit/logins, MCP limits, and any future IP-based guard. + +Suggested fix: + +- Prefer `cf-connecting-ip` on Cloudflare. +- Only trust `x-forwarded-for` from known proxy paths. +- Normalize and validate IP shape before using it as a rate-limit key. + +### P1 - Public npm stats requests can cause unbounded fan-out + +`src/utils/stats-queries.functions.ts:12-33` uses typed pass-through validators for public bulk npm stats input. `src/utils/stats.server.ts:470-646` turns those values into per-package, per-range chunk requests and runs missing chunks with `Promise.all` at `src/utils/stats.server.ts:549-646`. + +The stats UI route schemas do not cap package counts, string length, or date span tightly enough. Examples include `src/routes/stats/npm/-comparisons.ts:3-13`, `src/routes/stats/npm/-utils.ts:44-47`, `src/routes/stats/npm/index.tsx:57-86`, and `src/routes/stats/npm/$packages.tsx:15-23`. A public request can fan out to many npm API requests, Blob storage listings, and DB cache operations. + +`src/utils/npm-download-cache.server.ts:368-390` also lists all Blob objects under a prefix with page size `1000` but no page/object cap. That is called from latest-chunk lookups at `src/utils/npm-download-cache.server.ts:560-607` and `src/utils/npm-download-cache.server.ts:636-691`, so a large package set can turn into many unbounded storage-list operations. + +The DB fallback has the same scaling shape. `src/utils/stats-db.server.ts:1347-1382` and `src/utils/stats-db.server.ts:1422-1459` build one `or(...)` predicate per requested chunk, so uncapped input can create a huge SQL predicate before npm fetches even start. The newer Blob helper also runs storage/cache reads with unbounded `Promise.all` at `src/utils/npm-download-cache.server.ts:506-555`, `src/utils/npm-download-cache.server.ts:560-631`, and `src/utils/npm-download-cache.server.ts:636-715`. + +The authenticated MCP npm stats tool caps `packages` at 10 in `src/mcp/tools/npm-stats.ts:19-43`, but package/library/preset strings are still loose and flow into the same stats code at `src/mcp/tools/npm-stats.ts:163-176`. + +Suggested fix: + +- Add a shared runtime schema for npm stats queries. +- Cap package group count, packages per group, package name length/pattern, and max date window. +- Add concurrency limiting for npm fetches. +- Cap cache batch sizes and split large chunk lookups into bounded pages. +- Add page/object caps to npm download-cache Blob listings. +- Add per-IP or per-session limits for public bulk stats calls. + +### P2 - NPM stats UI search state is unbounded too + +The backend fan-out issue is mirrored at the route/search layer. `src/routes/stats/npm/index.tsx:57-86` and `src/routes/_library/$libraryId/$version.docs.npm-stats.tsx:66-93` accept `packageGroups` arrays through URL search without max lengths, and `height` is just `v.number()` with no min/max. The slug redirect route parses unlimited package lists from `params.packages` at `src/routes/stats/npm/$packages.tsx:24-27` via `src/routes/stats/npm/-utils.ts:43-47`, then turns them into search `packageGroups` at `src/routes/stats/npm/$packages.tsx:91-101`. The shared `packageGroupSchema` at `src/routes/stats/npm/-comparisons.ts:3-13` also accepts arbitrary package names, color strings, and baseline labels. + +Result: a crafted stats URL can create a huge query string, oversized React Query key, large server-function payload, and a pathological chart container height before backend guards have a chance to help. + +Suggested fix: move npm stats route/search schemas into `src/components/npm-stats/shared.ts` or a dedicated `src/utils/npm-stats-schema.ts`, cap total packages/groups, validate npm package names, trim labels/colors, and clamp chart height to a sane range. + +### P2 - NPM stats baseline normalization can corrupt relative change + +`src/components/npm-stats/NPMStatsChart.tsx:219-233` captures `firstDownloads` before baseline normalization, then mutates each point's `d.downloads` in place when `normalizeByBaseline` is active. The returned `change` value subtracts the raw first download count from the normalized current value. + +That means the "Relative Change" transform can be wrong whenever baseline normalization is active: the y-value mixes two units. The in-place mutation also makes this block harder to reason about because the original binned point object is no longer raw after the first normalized pass. + +Suggested fix: compute `normalizedDownloads` as a local value, compute `firstNormalizedDownloads` with the same divisor policy, and return a fresh `{ ...d, downloads: normalizedDownloads, change: normalizedDownloads - firstNormalizedDownloads }` object without mutating the binned point. + +### P2 - Recent npm stats can return partial totals from partial cache hits + +`src/utils/npm-download-cache.server.ts:636-717` returns a map of the package names that have a covering cached chunk; it is intentionally partial when some packages miss both Blob and legacy cache. `src/utils/stats.server.ts:974-1023` treats `latestCachedChunks.size > 0` as a complete hit and returns aggregate daily/weekly/monthly totals using only the cached chunks. + +For a library with multiple packages, one cached package and one missing package can undercount all recent download totals while still looking fresh. + +Suggested fix: only take the fast path when `latestCachedChunks.size === packageNames.length`, or merge cached package results with bounded fetches for the missing package names before returning. + +### P1/P2 - Scheduled NPM stats refresh can hang forever on 429s + +The scheduled all-time package fetch loops until a chunk succeeds. `src/utils/stats.functions.ts:266-355` uses `while (!success)` and, for npm `429`, waits five seconds and retries the same chunk with no attempt cap, wall-clock deadline, or abort signal. `fetchSingleNpmPackageFresh` adds outer retries at `src/utils/stats.functions.ts:375-435`, but the inner 429 loop can prevent those retries from ever advancing. + +`computeNpmOrgStats` then runs package refreshes through an `AsyncQueuer` at `src/utils/stats.functions.ts:517-569`, so one stuck package chunk can hold the whole org refresh. The admin trigger reaches this path through `src/utils/stats-admin.server.ts:367-375`. + +Suggested fix: add a bounded retry policy for npm chunk fetches, use `AbortSignal.timeout`/a task deadline, record partial failures explicitly, and let the scheduled refresh finish with a degraded result instead of waiting indefinitely. + +### P1 - Builder API endpoints do not share request guards + +`src/routes/api/builder/compile.ts:6-32`, `compile-attributed.ts`, `validate.ts`, `suggest.ts`, `feature-artifacts.ts`, `download.ts`, `load-template.ts`, `load-remote-template.ts`, and `load-remote-addon.ts` read JSON/query input directly and do not share one request guard/schema. Some apply the builder rate-limit preset, but content-type, content-length, same-origin, filename, and response-error policy are still inconsistent. + +`RATE_LIMITS.builderCompile` exists in `src/utils/rateLimit.server.ts`, and `src/routes/api/application-starter/resolve.ts:80-166` already shows a stronger pattern with rate limits, body guards, content-type checks, same-origin checks, schema parsing, and cache headers. + +Specific risks: + +- `src/routes/api/builder/compile.ts:8-22` reads arbitrary JSON and only checks that `definition` exists. +- `src/routes/api/builder/download.ts:21-88` accepts raw query values and uses raw `name` for `zip.folder(name)` and `Content-Disposition`. +- Error responses expose internal exception messages in several builder endpoints. + +Suggested fix: + +- Create a shared builder request schema and parse every entrypoint through it. +- Apply `RATE_LIMITS.builderCompile` or a more specific preset. +- Add content-type, content-length, and same-origin guards. +- Sanitize project names and response filenames. +- Return stable public error codes and log internal details separately. + +### P1 - Remote builder loads have allowlist checks but no fetch budget + +`src/builder/api/remote.ts:101-104` fetches remote template JSON with no timeout, status check, content-type check, or response-size limit. `src/builder/api/remote.ts:125-127` delegates remote add-ons to `loadRemoteAddOn`. + +`src/utils/url-validation.server.ts` is a good SSRF/host allowlist, but allowed CDN URLs can still hang workers or return huge responses. The validation also happens before `fetch`; `src/builder/api/remote.ts:101-103` uses the default redirect-following behavior and does not revalidate the final response URL, so an allowed host redirect can bypass the initial host check. + +Suggested fix: + +- Add a central `fetchJsonWithLimit` helper with `AbortSignal.timeout`, byte cap, content-type guard, status guard, and JSON parse errors. +- Use `redirect: 'manual'` or revalidate every redirect/final URL against the same allowlist. +- Use it for remote templates and any remote add-on path that allows user-provided URLs. + +### P1 - GitHub docs webhook fails open when the secret is unset + +`src/routes/api/github/webhook.ts:67-85` only verifies the GitHub signature when `env.GITHUB_WEBHOOK_SECRET` is present. If the secret is missing, any public POST can send a watched repo/ref payload, mark docs/content cache rows stale at `src/routes/api/github/webhook.ts:136-139`, and trigger cache purge at `src/routes/api/github/webhook.ts:151`. + +Suggested fix: + +- Fail closed in production if `GITHUB_WEBHOOK_SECRET` is missing. +- Return a clear deploy/config error instead of accepting unsigned webhooks. +- Add a content-length cap before `request.text()`. +- Consider validating event id/delivery headers for replay diagnostics. + +### P1 - Intent package detail performs npm ingestion on the public request path + +`src/utils/intent.functions.ts:422-435` accepts any string package name and, when the package is missing locally, calls `inlineSeedPackage`. That path fetches npm metadata, downloads the latest tarball, extracts skills, writes package/version rows, and stores skill content at `src/utils/intent.functions.ts:304-396`. + +This turns an unauthenticated read endpoint into a package-ingestion worker. It also uses an unbounded in-memory `rejectedPackages` map keyed by arbitrary request strings at `src/utils/intent.functions.ts:247-260`. + +Suggested fix: + +- Validate package names with a shared npm package-name schema and length cap. +- Do not process unknown packages inline from public detail requests. +- Queue unknown packages for bounded background processing, or require admin/manual seeding. +- Add per-IP/session rate limits for public Intent registry misses. +- Cap or replace the local rejection cache with bounded cache infrastructure. + +### P1 - Intent tarball extraction has no time or size budget + +`src/utils/intent.server.ts:293-358` fetches a package tarball with no timeout, compressed-size cap, decompressed-size cap, entry-count cap, or per-file cap. Matching `SKILL.md` files are buffered fully at `src/utils/intent.server.ts:336-340`. + +Any npm package that reaches Intent discovery, admin seeding, scheduled queue processing, or the public inline-seed path can force large stream work. The scheduled GitHub discovery path also extracts a tarball inline before enqueueing at `src/server/scheduled.server.ts:230-331`, so the expensive tarball path is not isolated to the bounded queue processor. + +Suggested fix: + +- Fetch tarballs with an abort timeout and content-length cap. +- Enforce max decompressed bytes, max tar entries, max skill files, and max bytes per `SKILL.md`. +- Abort the pipeline as soon as any cap is exceeded. +- Store the caps in one reusable ingestion policy object. + +### P1/P2 - Public Intent registry queries lack request caps + +`src/utils/intent.functions.ts:104-240` accepts uncapped `search`, `framework`, `page`, and `pageSize`, then may call npm search and fetch versions/skills for every verified package. The default npm discovery helper also pages until npm's reported total ends at `src/utils/intent.server.ts:89-111`, with no page cap or deadline. `src/utils/intent.functions.ts:595-600` accepts uncapped skill-search limits. `src/utils/intent.functions.ts:697-735` accepts an uncapped `packageNames` array and fans out through `Promise.all`. The package-history paths have the same shape: `src/utils/intent.functions.ts:799-828` accepts an uncapped changelog `limit`, and `src/utils/intent.functions.ts:966-1007` walks every version for one skill with no limit. The package route also accepts unbounded `expanded` and `expandedSkills` URL arrays at `src/routes/intent/registry/$packageName.tsx:53-58`, then feeds them into sets and URL writes at `src/routes/intent/registry/$packageName.index.tsx:80-101` and `src/routes/intent/registry/$packageName.index.tsx:163-177`. + +Suggested fix: + +- Add shared public Intent query schemas with max query length, page size, page number, package count, package-name length, and history limit. +- Prefer precomputed directory rows for list pages instead of per-package version/skill lookups in the request path. +- Apply bounded concurrency where fan-out remains necessary. + +### P3 - Intent dependency graph uses a fixed SVG marker id + +`src/components/intent/SkillDependencyGraph.tsx:168-176` defines ``, and every dependency line references `markerEnd="url(#arrowhead)"` at `src/components/intent/SkillDependencyGraph.tsx:208-218`. If two dependency graphs ever render on the same page, the marker id collides across SVGs and references can resolve to the first definition. + +Suggested fix: generate the marker id with `React.useId()` and reference `url(#${markerId})`, or accept a stable id prefix from the parent if the graph needs deterministic screenshots. + +### P2/P3 - Intent package metadata URLs are rendered without URL normalization + +Intent package detail pages render package metadata from npm. `src/utils/intent.functions.ts:273-284` builds `repositoryUrl` from `latestMeta.repository` by stripping `git+` and `.git`, but does not parse the URL or restrict protocols. Directory rows also pass through npm-provided homepage/repository/npm links at `src/utils/intent.functions.ts:198-204`. + +The package layout renders `detail.repositoryUrl` directly as an external anchor at `src/routes/intent/registry/$packageName.tsx:257-264`. A package's npm metadata can contain non-web protocols or malformed values, and the UI still titles the link "GitHub". + +Suggested fix: normalize npm metadata URLs through a shared `normalizeExternalPackageUrl` helper that accepts only `https:` and `http:` for public anchors, converts common `git+https`/`git@github.com:` repository forms to web URLs, drops anything else, and stores a `repositoryHost`/label separately from the href. + +### P2/P3 - Intent skill source links trust tarball path segments + +`src/utils/intent.server.ts:319-334` accepts any tar entry matching `package/skills/**/SKILL.md` and derives `skillPath` by string replacement. It does not reject `..`, encoded separators, duplicate slashes, or odd path segments. The skill detail page then interpolates that value into an unpkg Source link at `src/routes/intent/registry/$packageName.$skillName.tsx:143-147`. + +The link is external, not a local file read, so this is lower risk than extraction traversal. It still lets package-controlled tar headers produce malformed or misleading source URLs on a trusted registry page. + +Suggested fix: parse tar entry names into normalized path segments before storing `skillPath`, reject traversal/empty segments, and build the unpkg URL with `new URL()` plus per-segment encoding. + +### P3 - NPM-style package slugs are not round-trip safe + +`decodePkgName` encodes scoped package names by replacing `/` with `__` at `src/routes/intent/registry/$packageName.tsx:29-33`. Directory links use the inverse shape with `pkg.name.replace('/', '__')` at `src/routes/intent/registry/index.tsx:556-563`, `src/routes/intent/registry/index.tsx:573-575`, and `src/routes/intent/registry/index.tsx:653-663`. + +The same codec exists for NPM stats at `src/routes/stats/npm/-utils.ts:34-40`, with parsed params used by `src/routes/stats/npm/$packages.tsx:24-26`. + +That delimiter is undocumented and ambiguous for any package name that already contains `__`; the route would decode the first delimiter into `/` and look up a different package. Suggested fix: use `encodeURIComponent`/`decodeURIComponent` for route params, or centralize an explicit package-name slug codec with tests for scoped, unscoped, underscore, and malformed names across Intent and stats routes. + +### P3 - Maintainer social links rely on raw static URL strings + +Maintainer social profile data is typed as loose strings at `src/libraries/maintainers.ts:15-20`, then rendered directly as anchor hrefs by `src/components/MaintainerCard.tsx:183-200`. One current entry already misses a protocol: `src/libraries/maintainers.ts:316-319` sets `website: 'harry-whorlow.dev'`, which renders as a relative site URL instead of an external profile link. + +Suggested fix: validate static maintainer data at module load/build time or normalize it through the same external URL helper used for package metadata. The helper should either add `https://` for known bare domains or reject malformed profile URLs loudly. + +## Correctness And Stability Findings + +### P2 - Game persistence runs from module scope + +`src/components/game/hooks/useGameStore.ts:48-53` casts parsed `localStorage` JSON to `PersistedState` without runtime validation. `src/components/game/hooks/useGameStore.ts:1000-1016` starts an interval and `beforeunload` listener at module scope when `window` exists. + +This is easy to duplicate under HMR and hard to clean up. Move persistence into a mounted component/hook with teardown and schema validation. + +### P2 - Game engine async initialization can attach work after disposal + +`src/components/game/scene/VanillaGameScene.tsx:64-80` creates a `GameEngine`, calls async `engine.init()`, and disposes the engine on unmount. `GameEngine.start()` guards `isDisposed` at `src/components/game/engine/GameEngine.ts:700-710`, but `GameEngine.init()` itself continues after asset preload at `src/components/game/engine/GameEngine.ts:172-217` and can add scene objects, store subscriptions, and `BoatControlSystem` listeners after disposal. + +The engine has the same issue after setup. `src/components/game/engine/GameEngine.ts:391-421` and `src/components/game/engine/GameEngine.ts:576-606` fetch showcase data and then mutate store/entity/ocean state from `.then(...)` callbacks without checking `this.isDisposed`. `src/components/game/engine/entities/Islands.ts:1034-1050` loads partner logo textures and adds meshes in the `TextureLoader.load` callback without knowing whether the island/info group has already been disposed. `src/components/game/engine/GameEngine.ts:674-687` also schedules an untracked `setTimeout` inside a store subscription. If the engine is disposed before any of those callbacks fire, stale work can still spawn AI or update disposed scene objects. + +Smaller UI timer examples: + +- `src/components/game/ui/BadgeOverlay.tsx:112-115` schedules dismiss work without cleanup. +- `src/components/game/ui/TouchControls.tsx:4-10` uses module-scope click debounce state and an untracked timeout. + +Suggested fix: + +- Add an abort/cancel guard to `GameEngine.init()` after every awaited preload step. +- Track and clear engine-owned timeouts in `dispose()`. +- Avoid adding systems/subscriptions/listeners after `isDisposed` is true. +- Consider making `init()` return a cleanup-capable task or accepting an `AbortSignal`. + +### P3 - Game discovery confetti never uses island colors + +`src/components/game/engine/entities/Islands.ts:933-949` creates the flag material with the island color but never stores that color on `flag.userData`. `spawnConfetti` reads `instance.flagGroup.userData.color || '#FFD700'` at `src/components/game/engine/entities/Islands.ts:1389-1396`, so every island falls back to gold-accent confetti instead of deriving its palette from the discovered library/partner/showcase. + +Suggested fix: set `flag.userData.color = color` when the flag is created, or pass the island color through the `IslandInstance` data instead of userData. + +### P2/P3 - Game island culling misses world-space children + +`src/components/game/engine/entities/Islands.ts:1262-1268` distance-culls only `instance.group`. Several visible children are intentionally added directly to the top-level `Islands.group` instead of `instance.group`: lobe wave rings at `src/components/game/engine/entities/Islands.ts:253-273`, flags at `src/components/game/engine/entities/Islands.ts:422-431`, info cards at `src/components/game/engine/entities/Islands.ts:433-437`, and main wave rings at `src/components/game/engine/entities/Islands.ts:439-462`. + +The update loop skips wave-ring animation when `instance.group.visible` is false at `src/components/game/engine/entities/Islands.ts:1369-1375`, but it never hides those meshes. Distant islands can therefore still render their world-space rings/flags/cards while the land mass is culled. Suggested fix: add a per-instance world-space group for rings/flag/info and cull it together, or explicitly update visibility for every detached child. + +### P3 - Island flag position ignores the rotation it computes + +`calculateFlagWorldPosition` computes rotated local offsets at `src/components/game/engine/entities/Islands.ts:963-972`, but returns unrotated `signOffsetX` and `signOffsetZ` at `src/components/game/engine/entities/Islands.ts:974-978`. Rotated islands therefore place flags at the same world offset instead of the rotated local flag anchor. Suggested fix: return `data.position[0] + localX * data.scale` and `data.position[2] + localZ * data.scale`, with any intentional fixed nudge named separately. + +### P2 - Progression machine checks completion before counting the current discovery + +The progression machine guards compare current context counts against totals at `src/components/game/machines/progressionMachine.ts:117-128`, but the transition actions increment only after the guard is evaluated. The discovery transitions repeat that shape for libraries, partners, showcases, and corners at `src/components/game/machines/progressionMachine.ts:259-265`, `src/components/game/machines/progressionMachine.ts:288-294`, `src/components/game/machines/progressionMachine.ts:304-310`, and `src/components/game/machines/progressionMachine.ts:320-326`. + +The sync path sends exactly one event for the newly discovered island at `src/components/game/machines/useProgressionSync.ts:34-67` after `discoverIsland` adds the id to the store at `src/components/game/hooks/useGameStore.ts:399-415`. If the machine has `total - 1` discoveries and the player discovers the last island, the guard sees the old count, takes the non-transition branch, increments to `total`, and then waits for another discovery event that may never happen. Zustand still unlocks stages directly at `src/components/game/hooks/useGameStore.ts:421-489`, so UI/gameplay and badge machine state can diverge. + +Suggested fix: make the completion guard include the incoming discovery (`context.count + 1 >= total`), or increment first and use `always` transitions to check completion from updated context. Add regression tests for `1/1` and `N/N` completions per phase. + +### P2 - Restored game progress misclassifies core library islands + +`GameMachineProvider` restores machine counts from `discoveredIslands` by checking id prefixes at `src/components/game/machines/GameMachineProvider.tsx:96-127`: `library-`, `partner-`, `showcase-`, and `corner-`. Core library islands are generated with the raw library id at `src/components/game/utils/islandGenerator.ts:162-168`, not a `library-*` prefix, so restored `librariesDiscovered` can stay at `0` after reload even when the player has discovered Query/Table/Router/etc. + +That can leave badge/progression machine state behind the zustand gameplay state. Suggested fix: classify discoveries from the actual island collections, or share the same island-type lookup used by live discovery sync instead of inferring type from string prefixes. + +### P2 - Ocean rock regeneration reuses disposed shared geometry + +`OceanRocks` creates one shared `dodecahedronGeo` at `src/components/game/engine/entities/OceanRocks.ts:35-38`. `generate()` clears the previous rock set by calling `dispose()` at `src/components/game/engine/entities/OceanRocks.ts:50-53`, and `dispose()` disposes each `rock.rockMesh.geometry` at `src/components/game/engine/entities/OceanRocks.ts:210-217`. Those meshes all point at the shared `dodecahedronGeo`, but `generate()` then immediately creates new rock meshes with the same disposed geometry at `src/components/game/engine/entities/OceanRocks.ts:105-120`. + +Today `GameEngine.initializeGameData()` calls `generate()` once at `src/components/game/engine/GameEngine.ts:336-357`, but the method is written as a reusable regeneration API and will break under HMR, reset/regenerate features, or any future world refresh. + +Suggested fix: treat `dodecahedronGeo` like an owner-level resource and dispose it only from a final `dispose()` path, or recreate it after clearing generated groups. Per-rock cleanup should dispose only per-rock materials and cached ring geometries that are not reused. + +### P2 - Game model cleanup disposes cached/shared geometry + +`modelLoader.clone()` clones the cached scene at `src/components/game/engine/loaders/ModelLoader.ts:43-50` and clones only the material at `src/components/game/engine/loaders/ModelLoader.ts:56-58`. Geometry stays shared with the cached GLTF. `Boat.dispose()` then traverses the current boat model and disposes `child.geometry` at `src/components/game/engine/entities/Boat.ts:291-298`, and `AIShips.disposeShip()` does the same at `src/components/game/engine/entities/AIShips.ts:146-154`. + +That means disposing one boat/AI clone can dispose geometry still owned by the loader cache or by other clones. `AIShips` has the same problem for its own cannon geometries: `createShip()` uses shared `this.cannonGeometries.base/barrel` at `src/components/game/engine/entities/AIShips.ts:49-64`, but `disposeShip()` disposes each child geometry when one AI ship is removed. The next ship can render with a disposed shared cannon geometry. + +There is also an opposite leak in the same area: `Boat.setBoatType()` removes the old model from the group at `src/components/game/engine/entities/Boat.ts:75-83` but does not dispose its cloned materials before replacing it. + +Suggested fix: make ownership explicit. If model geometry is shared through `ModelLoader`, instance cleanup should dispose only cloned materials/textures, not geometry. Shared cannon geometry should only be disposed in `AIShips.dispose()`. If per-instance geometry is required, deep-clone geometry at creation and keep the disposal contract local. + +### P2 - Health damage flash can retrigger forever after one hit + +`GameHUD` tracks `prevHealth` separately from `boatHealth` at `src/components/game/ui/GameHUD.tsx:44-45`. The damage-flash effect detects a drop at `src/components/game/ui/GameHUD.tsx:106-114`, sets `damageFlash`, and returns without updating `prevHealth`. After the 150ms timer clears `damageFlash`, `boatHealth` is still lower than the stale `prevHealth`, so the effect can schedule another flash for the same old hit. The bar can keep flashing until health is restored or another path updates `prevHealth`. + +Suggested fix: update `prevHealth` whenever a new `boatHealth` value is observed, including the damage branch. A simple pattern is to compare against the previous value in a functional state update, trigger the flash as a side effect of the comparison, and always store the current health for the next render. + +### P2/P3 - Game generated vectors can become NaN or infinite + +`createAICannonball` computes a lead target at `src/components/game/engine/systems/AISystem.ts:314-316` and divides by `leadDist` at `src/components/game/engine/systems/AISystem.ts:318-319` without a zero guard. If the target point equals the cannon fire point after spread/lead math, the cannonball velocity becomes `NaN`, and that value is written into the shared cannonball array. + +Island placement has the same shape. `generateIslands` pushes islands apart by dividing by `dist` at `src/components/game/utils/islandGenerator.ts:107-113`, and the expanded/showcase generators repeat it at `src/components/game/utils/islandGenerator.ts:315-324` and `src/components/game/utils/islandGenerator.ts:417-425`. Duplicate generated positions are unlikely, but if they happen the world coordinates can become `Infinity`/`NaN` and then flow into collisions, minimap, ocean gradients, and Three transforms. + +Suggested fix: centralize a `safeNormalize2D(dx, dz, fallbackAngle)` helper for AI fire, collision push, and generator push-apart math. When distance is zero or non-finite, use a deterministic fallback vector from the seed/id instead of dividing. + +### P3 - Compass shop item ignores showcase and corner islands + +The active boat control system correctly includes `showcaseIslands` and `cornerIslands` in late-game collision/nearby detection at `src/components/game/engine/systems/BoatControlSystem.ts:152-156`. The compass purchase path does not: `purchaseItem('compass')` builds `allIslands` from only `islands` and `expandedIslands` at `src/components/game/hooks/useGameStore.ts:787-797`. + +After showcase or corner islands unlock, a bought compass can only target core library or partner islands, so it stops helping with the current progression layer. Suggested fix: extract one `getReachableIslands(state)` helper and use it in controls, compass targeting, restart spawn selection, minimap, and counters. + +The AI system has the same split in a different behavior. `src/components/game/engine/systems/AISystem.ts:409-424` builds obstacle-avoidance islands from only `islands` and `expandedIslands`, then uses that list at `src/components/game/engine/systems/AISystem.ts:554-565`. Outer-rim and boss ships can therefore ignore showcase/corner island collision pressure even though the player collides with those islands. Include AI avoidance in the shared reachable-islands helper, or expose separate `getCollidableIslands(state)` / `getCompassTargetableIslands(state)` helpers if the sets intentionally differ. + +### P2 - Custom `useMutation` has stale callbacks and casts through redirects + +`src/hooks/useMutation.ts:29-53` only depends on `opts.fn`, while using `opts.onSuccess`, `opts.onError`, and `opts.onSettled`. Changed callbacks can go stale. The catch path does not await `onSettled` even though the type allows a promise. `src/hooks/useMutation.ts:59` and `src/hooks/useMutation.ts:86-88` use casts. + +Consider replacing this with `@tanstack/react-query` mutation helpers or tighten this hook with refs/deps and typed redirect handling. + +### P2 - Tooltip clone drops/overrides child behavior + +`src/components/Tooltip.tsx:43-47` clones the child with a new `ref` and `getReferenceProps()` but does not merge the child ref or pass existing child props into `getReferenceProps`. + +Use the Floating UI pattern: `getReferenceProps(children.props)` and a merged ref utility. + +### P2/P3 - Button and select primitives rely on native submit defaults + +`src/ui/Button.tsx:129-144` creates the requested element and forwards props, but when `as` is omitted it renders a native `