Skip to content

fix(tables): route large CSV imports to the background job instead of 413#4927

Merged
TheodoreSpeaks merged 3 commits into
stagingfrom
fix/import-job-failure
Jun 9, 2026
Merged

fix(tables): route large CSV imports to the background job instead of 413#4927
TheodoreSpeaks merged 3 commits into
stagingfrom
fix/import-job-failure

Conversation

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator

Summary

  • Large CSV imports (≥8MB) were 413ing instead of using the background job. A prior refactor (improvement(platform): remove tour, simplify sidebar/header, drop loading skeletons #4354) dropped the file-size dispatch from both import entry points, leaving every import on the synchronous, 10MB-proxy-capped routes.
  • Re-wired the size-based dispatch: large files now upload straight to storage (bypassing the proxy cap) and import via the existing background worker; small files keep the fast sync path.
  • Restored the 512KB preview-slice parse in the import dialog so large files aren't fully parsed client-side (main-thread jank / V8 string-ceiling crashes). Row-limit enforcement happens server-side via the DB trigger.
  • Restored the ImportProgressMenu tray in the tables list view for live background-import progress.
  • Background imports are tracked only by the tray (header upload button no longer double-shows 0/1); plain "importing in the background" toast.

Type of Change

  • Bug fix

Testing

Tested manually — imported a 10MB CSV (background job kicks off, no 413, tray shows progress) and a small CSV (sync path, header counter). bun run lint and bun run check:api-validation:strict pass.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jun 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jun 9, 2026 6:21pm

Request Review

@cursor

cursor Bot commented Jun 9, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches data-import paths and upload/cancel behavior for large files; wrong threshold routing could still 413 or skip background import, but changes reuse existing async hooks and server enforcement.

Overview
Restores size-based routing for table CSV imports: files at or above CSV_ASYNC_IMPORT_THRESHOLD_BYTES (~8MB) upload direct-to-storage and run through the background import worker, avoiding synchronous POSTs that hit the proxy 413 body cap. Smaller files keep the existing sync import path.

In ImportCsvDialog, file selection only parses a 512KB preview for headers/mapping (not the full file), and large imports close the dialog immediately, show the import tray with upload progress, and support mid-upload cancel via cancelTableImport. Client-side row-limit capacity warnings are removed; limits are enforced server-side.

On the tables list, bulk CSV upload splits large vs small files: async imports use useImportCsvAsync and the tray (not the header 0/1 upload counter), and ImportProgressMenu is wired back into the list header for live background-import status.

Reviewed by Cursor Bugbot for commit a821d4c. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit fcc28f7. Configure here.

Comment thread apps/sim/app/workspace/[workspaceId]/tables/tables.tsx Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

@greptile-apps

greptile-apps Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR restores size-based dispatch for CSV imports that was dropped in a prior refactor, fixing 413 errors for files ≥8 MB. Large files are uploaded directly to storage (bypassing the 10 MB proxy cap) and imported via the existing background worker; small files keep the synchronous POST path unchanged.

  • import-csv-dialog.tsx: Adds useImportCsvIntoTableAsync, previews only the first 512 KB (avoiding V8-string ceiling crashes on huge files), removes client-side row-limit checks (now enforced by a DB trigger server-side), and wires the ImportTrayStore for upload progress and cancel.
  • tables.tsx: Re-introduces the split in the header file-picker (asyncFiles / syncFiles), tracks in-flight large uploads in the ImportTrayStore under a temporary pending_* key, and restores the ImportProgressMenu tray. The importId guard before cancelTableImport is also fixed here.

Confidence Score: 5/5

Safe to merge — the bug fix is well-contained, the async dispatch path reuses the existing background worker, and the sync path is unchanged.

The size-based routing logic is straightforward, the tray/cancel wiring mirrors the pattern already used elsewhere in the codebase, and the removal of client-side row-limit checks is intentional (server enforces via DB trigger). No incorrect state transitions, missing guards, or data-loss paths were found.

No files require special attention; the two suggestions in import-csv-dialog.tsx are minor correctness polish.

Important Files Changed

Filename Overview
apps/sim/app/workspace/[workspaceId]/tables/components/import-csv-dialog/import-csv-dialog.tsx Adds async import path for large files (≥8MB): routes to background storage upload + worker, restores 512KB preview slice to avoid main-thread parse, removes client-side row-limit checks (server enforces via DB trigger), and wires the ImportTrayStore for progress/cancel.
apps/sim/app/workspace/[workspaceId]/tables/tables.tsx Re-adds size-based dispatch in the header file-picker: large files go via useImportCsvAsync + ImportTrayStore with a pending_* key, small files keep the synchronous upload path; restores ImportProgressMenu tray and fixes importId guard before cancelTableImport.

Reviews (3): Last reviewed commit: "fix(tables): guard importId on async can..." | Re-trigger Greptile

Comment thread apps/sim/app/workspace/[workspaceId]/tables/tables.tsx
Comment thread apps/sim/app/workspace/[workspaceId]/tables/tables.tsx Outdated
Comment thread apps/sim/app/workspace/[workspaceId]/tables/tables.tsx Outdated
@greptile-apps

greptile-apps Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR re-introduces the size-based dispatch that was dropped in a prior refactor: files ≥ 8 MB are uploaded directly to storage and imported via the background worker (bypassing the 10 MB proxy cap), while smaller files continue to use the synchronous path. The preview parse is capped at 512 KB so large files are never fully read on the main thread, and ImportProgressMenu is restored to the tables list for live background-import progress.

  • import-csv-dialog.tsx: adds useImportCsvIntoTableAsync, a 512 KB preview slice (parseCsvPreview), and a size gate in handleSubmit that seeds the tray, closes the dialog, and fires the background mutation without blocking the UI. Row-capacity pre-checks are removed; the DB trigger enforces the limit server-side.
  • tables.tsx: splits the header upload button's handleCsvChange into async and sync loops, uses pending tray IDs (pending_<generateId>) until the real table ID is known from the kickoff response, restores ImportProgressMenu via leadingActions, and wires cancelTableImport for mid-upload cancellation.

Confidence Score: 3/5

The core routing fix is solid, but the async error path surfaces two toast notifications on failure in both changed files.

The double-toast bug is present on both the list-view upload and the dialog async path: when an upload fails, users see two separate error messages every time. This is a real, reproducible defect on the new code path.

tables.tsx — the catch block inside the asyncFiles loop and the mutation's onError both call toast.error; import-csv-dialog.tsx — the inline onError passed to importAsyncMutation.mutate and the hook's onError do the same.

Important Files Changed

Filename Overview
apps/sim/app/workspace/[workspaceId]/tables/components/import-csv-dialog/import-csv-dialog.tsx Adds async large-file path: file sliced to 512 KB for preview, row-limit pre-checks removed (delegated to server DB trigger), dialog dispatches background import and closes immediately. Double-toast on async error shared with tables.tsx.
apps/sim/app/workspace/[workspaceId]/tables/tables.tsx Splits CSV uploads into async (≥8MB tray-tracked) and sync paths; restores ImportProgressMenu. Two issues: mutation objects in useCallback deps, and double error toast when async upload fails.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User selects CSV] --> B{File size}
    B -->|ge 8 MB async path| C[Upload direct to storage]
    B -->|lt 8 MB sync path| D[POST via proxy]
    C --> C1[Kickoff background worker]
    C1 --> C2[ImportProgressMenu tray tracks progress]
    C1 --> C4{Error?}
    C4 -->|Yes| C5[endUpload + double toast warning]
    C4 -->|No| C2
    D --> D2{Error?}
    D2 -->|Yes| D3[submitError in dialog]
    D2 -->|No| D4[Toast success]
    A2[Open ImportCsvDialog] --> P1[parseCsvPreview 512 KB slice]
    P1 --> P2[Show mapping UI]
    P2 --> B2{File size}
    B2 -->|ge 8 MB| C
    B2 -->|lt 8 MB| D
Loading

Reviews (2): Last reviewed commit: "fix(tables): drop duplicate error toast ..." | Re-trigger Greptile

Comment thread apps/sim/app/workspace/[workspaceId]/tables/tables.tsx
Comment thread apps/sim/app/workspace/[workspaceId]/tables/tables.tsx
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile review

@TheodoreSpeaks TheodoreSpeaks merged commit 2f70188 into staging Jun 9, 2026
14 checks passed
@TheodoreSpeaks TheodoreSpeaks deleted the fix/import-job-failure branch June 9, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant