feat: add SQL transpilation for pre-aggregate sql_filter expressions#21734
feat: add SQL transpilation for pre-aggregate sql_filter expressions#21734IrakliJani wants to merge 1 commit intomainfrom
Conversation
This stack of pull requests is managed by Graphite. Learn more about stacking. |
There was a problem hiding this comment.
revert this file :) or just use it for testing
🧪 Test Selection✅ Tests that will run
⏭️ Tests skipped (no relevant file changes detected)
|
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
|
Your preview environment pr-21734 has been deployed. Preview environment endpoints are available at: |
Preview Environment🌐 URL: https://lightdash-preview-pr-21734.lightdash.okteto.dev 📋 Logs: View in GCP Console 🔧 SSH: |

Description:
This PR adds SQL transpilation support for pre-aggregates to handle cross-database compatibility when the source warehouse uses a different SQL dialect than DuckDB.
Key Changes:
Added
@polyglot-sql/sdkdependency for SQL dialect transpilation between different database systems (Snowflake, BigQuery, Postgres, etc. → DuckDB)Implemented
transpileExploreSqlFiltersfunction that:sqlWhereexpressions from source warehouse dialects to DuckDB syntaxstatus→orders_status)${lightdash.attributes.X}and${ld.attr.X}Enhanced pre-aggregate matching to allow matches when base tables have
sqlWherefilters, since they can now be properly transpiled at query timeUpdated demo configuration with a comprehensive
sql_filterexample showing attribute-based filtering with comma-separated values and PostgreSQL-specific syntaxAdded comprehensive test coverage for various SQL transpilation scenarios including cross-table joins, complex date filters, and edge cases
The transpilation ensures that pre-aggregates work seamlessly across different warehouse types while maintaining the same filtering logic and user attribute support.