Skip to content

feat(tables): upload csvs #3607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
icecrasher321 merged 5 commits into from
Mar 16, 2026
Merged

feat(tables): upload csvs #3607

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
de371f8
feat(tables): upload csvs
icecrasher321 Mar 16, 2026
6802bf7
address comments
icecrasher321 Mar 16, 2026
5ad358c
address comments
icecrasher321 Mar 16, 2026
2ccd18c
user id attribution
icecrasher321 Mar 16, 2026
df91819
fix boolean coercion
icecrasher321 Mar 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
address comments
  • Loading branch information
@icecrasher321
icecrasher321 committed Mar 16, 2026
commit 6802bf7740f08c9fc5012e970ab7df56a8422d73
21 changes: 17 additions & 4 deletions apps/sim/app/api/table/import-csv/route.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@ import { normalizeColumn } from '@/app/api/table/utils'

const logger = createLogger('TableImportCSV')

const MAX_CSV_FILE_SIZE = 50 * 1024 * 1024
const MAX_BATCH_SIZE = 1000
const SCHEMA_SAMPLE_SIZE = 100

type ColumnType = 'string' | 'number' | 'boolean' | 'date'

async function parseCsvBuffer(
buffer: Buffer
buffer: Buffer,
delimiter = ','
): Promise<{ headers: string[]; rows: Record<string, unknown>[] }> {
const { parse } = await import('csv-parse/sync')
const parsed = parse(buffer.toString('utf-8'), {
Expand All @@ -31,6 +33,7 @@ async function parseCsvBuffer(
relax_quotes: true,
skip_records_with_error: true,
cast: false,
delimiter,
}) as Record<string, unknown>[]

if (parsed.length === 0) {
Expand Down Expand Up @@ -121,8 +124,10 @@ function coerceValue(value: unknown, colType: ColumnType): string | number | boo
const s = String(value).toLowerCase()
return s === 'true'
}
Comment thread
icecrasher321 marked this conversation as resolved.
case 'date':
return new Date(String(value)).toISOString()
case 'date': {
const d = new Date(String(value))
return Number.isNaN(d.getTime()) ? String(value) : d.toISOString()
}
default:
return String(value)
}
Expand Down Expand Up @@ -164,6 +169,13 @@ export async function POST(request: NextRequest) {
return NextResponse.json({ error: 'CSV file is required' }, { status: 400 })
}
Comment thread
icecrasher321 marked this conversation as resolved.

if (file.size > MAX_CSV_FILE_SIZE) {
return NextResponse.json(
{ error: `File exceeds maximum allowed size of ${MAX_CSV_FILE_SIZE / (1024 * 1024)} MB` },
{ status: 400 }
)
}

if (!workspaceId) {
return NextResponse.json({ error: 'Workspace ID is required' }, { status: 400 })
}
Expand All @@ -179,7 +191,8 @@ export async function POST(request: NextRequest) {
}
Comment thread
icecrasher321 marked this conversation as resolved.

const buffer = Buffer.from(await file.arrayBuffer())
const { headers, rows } = await parseCsvBuffer(buffer)
const delimiter = ext === 'tsv' ? '\t' : ','
const { headers, rows } = await parseCsvBuffer(buffer, delimiter)

const columns = inferSchema(headers, rows)
const headerToColumn = new Map(headers.map((h, i) => [h, columns[i].name]))
Expand Down
14 changes: 13 additions & 1 deletion apps/sim/app/workspace/[workspaceId]/tables/tables.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
ModalContent,
ModalFooter,
ModalHeader,
toast,
Upload,
} from '@/components/emcn'
import { Columns3, Rows3, Table as TableIcon } from '@/components/emcn/icons'
Expand Down Expand Up @@ -171,11 +172,12 @@ export function Tables() {
})

if (csvFiles.length === 0) {
logger.warn('No CSV/TSV files selected')
toast.error('No CSV or TSV files selected')
return
}

setUploadProgress({ completed: 0, total: csvFiles.length })
const failed: string[] = []

for (let i = 0; i < csvFiles.length; i++) {
try {
Expand All @@ -189,11 +191,21 @@ export function Tables() {
}
}
} catch (err) {
failed.push(csvFiles[i].name)
logger.error('Error uploading CSV:', err)
}
Comment thread
icecrasher321 marked this conversation as resolved.
Comment thread
icecrasher321 marked this conversation as resolved.
}
Comment thread
icecrasher321 marked this conversation as resolved.

if (failed.length > 0) {
toast.error(
failed.length === 1
? `Failed to import ${failed[0]}`
: `Failed to import ${failed.length} file${failed.length > 1 ? 's' : ''}: ${failed.join(', ')}`
)
}
} catch (err) {
logger.error('Error uploading CSV:', err)
toast.error('Failed to import CSV')
} finally {
setUploading(false)
setUploadProgress({ completed: 0, total: 0 })
Expand Down
Loading