clickhouse: type inference via the xqlc analysis core#4521
Draft
kyleconroy wants to merge 3 commits into
Draft
Conversation
Port xqlc's core catalog (SQLite-backed sql_* catalog) and its dialect-neutral query analyzer into internal/core, repointing the analyzer from xqlc's copy of the AST onto sqlc's internal/sql/ast so there is a single AST. No converter and no second AST package. A smoke test drives the analyzer with sqlc's own PostgreSQL parser to prove the repointed analyzer resolves columns, types, star expansion, and aliases end-to-end against internal/sql/ast. This is the first step of merging xqlc back into sqlc as the future analysis core; ClickHouse will be the first engine wired onto it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XTGxNHW6v1S1YyC9FDSgrK
Wire ClickHouse onto the merged core: a dialect seed registering the built-in ClickHouse types, and a DDL handler that populates the core catalog from CREATE TABLE using sqlc's existing ClickHouse parser. A smoke test proves the full vertical path — ClickHouse SQL -> sqlc's ClickHouse parser -> internal/sql/ast -> core catalog + analyzer -> PrepareResult — resolving column names, types, nullability, source bindings, and star expansion, with none of the legacy compiler analyze step involved. Also fix the ClickHouse converter to render nested type parameters (the inner type of Nullable(T)/Array(T), Decimal precision, etc.) into TypeName.Name instead of dropping them as TODO nodes, so wrapped types resolve to their effective scalar type. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XTGxNHW6v1S1YyC9FDSgrK
Add the EngineClickHouse engine and a dedicated compile path so `sqlc generate` produces Go for ClickHouse entirely through the xqlc core, bypassing the legacy compiler analyze step and the in-memory sql/catalog: - config: add the "clickhouse" engine constant. - compiler: NewCompiler builds a core.Catalog seeded with the ClickHouse dialect; parseCatalog applies schema DDL to it; a new parseQueryCore resolves each query's columns and parameters via core/analyzer and assembles *compiler.Query, reusing only the shared query-metadata parsing. The legacy analyzeQuery/inferQuery/outputColumns path and the analyzer.Analyzer seam are never entered. - codegen: project the core catalog into plugin.Catalog for model/enum generation, and add a ClickHouse -> Go type map (Nullable(T) -> *T, the integer ladder, Float32/64, String, DateTime -> time.Time, ...). - clickhouse parser: compute statement byte-spans with a running offset and a semicolon scan (doubleclick reports statement starts but not ends), so leading "-- name:" annotations fall inside each statement. An endtoend case (clickhouse_select) exercises the full pipeline and its golden Go output is committed. Updating parse_basic/clickhouse's golden reflects the corrected statement spans and now-detected query name/cmd. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XTGxNHW6v1S1YyC9FDSgrK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
sqlc already has a working ClickHouse parser (
internal/engine/clickhouse, backed bydoubleclick), but no analysis for it: the ClickHouse catalog was a stub, the engine was not wired into the compiler, and there was noEngineClickHouse. The missing layer is a typed catalog + a query analyzer.This PR merges the xqlc analysis core into sqlc as the future analysis engine, and makes ClickHouse the first engine to run on it, end to end. ClickHouse is a natural first target because it has a parser and zero existing analysis — this is net-new, with no regression surface for the other engines.
What's here
sqlc generatenow works for ClickHouse, resolving column and parameter types entirely through the xqlc core.1.
core: land xqlc catalog + analyzer on sqlc's ASTinternal/core/— xqlc's SQLite-backed catalog (sql_*tables modeled on the Postgres system catalogs).internal/core/analyzer/— xqlc's dialect-neutral analyzer, repointed off xqlc's copy of the AST onto sqlc'sinternal/sql/ast. Single AST — no second copy, no converter.2.
clickhouse: analyze queries through the core catalog + analyzerseed.go(ClickHouse dialect + built-in types) andschema.go(DDL → core catalog).Nullable(String),Array(UInt64),Decimal(18,4)) instead of dropping them asTODO.3.
clickhouse: wire end-to-end sqlc generate onto the coreconfig: theclickhouseengine constant.compiler:NewCompilerbuilds a core catalog seeded with the ClickHouse dialect;parseCatalogapplies schema DDL to it; a dedicatedparseQueryCoreresolves columns/params viacore/analyzerand assembles*compiler.Query, reusing only the shared query-metadata parsing.codegen: a core-catalog →plugin.Catalogprojection for models/enums, plus a ClickHouse → Go type map (Nullable(T)→*T, the integer ladder,DateTime→time.Time, …).clickhouse parser: computes statement byte-spans (doubleclick reports statement starts but not ends), so-- name:annotations fall inside each statement.endtoendcase (clickhouse_select) exercises the full pipeline; its generated Go is committed as golden output.Example generated output for
SELECT id, name, tag, amount, created FROM eventsoverid UInt64, name String, tag Nullable(String), amount Float64, created DateTime:Design decisions
internal/sql/ast; the analyzer was repointed, not duplicated.internal/sql/catalogis not used for ClickHouse.analyzeQuery/inferQuery/outputColumnsor theanalyzer.Analyzerinterface.Follow-ups
core/analyzer(INSERT/UPDATE/DELETE) and bind-parameter detection for ClickHouse?placeholders.decimal.Decimal,uuid.UUID,netip.Addr,json.RawMessage) — these need import wiring; they currently resolve tostring. Array-ness is not yet modeled on catalog attributes.testgen-style conformance suite against a real ClickHouse.Still a draft: this is the first engine on the new core and the coverage above is intentionally scoped. When other engines migrate, they follow the same path and the legacy analyze machinery can be retired.
go build ./...andgo vetpass; new unit tests and theclickhouse_select/parse_basic/clickhouseendtoend cases are green.🤖 Generated with Claude Code
https://claude.ai/code/session_01XTGxNHW6v1S1YyC9FDSgrK