Context
PR #5827 added dbt integration that requires a timestamp field for point-in-time correctness.
Problem
The code validates that the timestamp field exists in the model columns but doesn't validate that it's actually a timestamp type. This could lead to runtime errors or incorrect behavior.
Current Behavior
# In dbt_import.py:183-189
if timestamp_field not in column_names:
click.echo(
f"{Fore.YELLOW}Warning: Model '{model.name}' missing timestamp "
f"field '{timestamp_field}'. Skipping.{Style.RESET_ALL}"
)
continueNo type checking is performed.
Proposed Solution
Add validation that the timestamp field is one of the timestamp types:
- TIMESTAMP
- TIMESTAMP_NTZ / TIMESTAMP_LTZ / TIMESTAMP_TZ
- DATETIME
- DATE
Warn users if the field exists but has an incompatible type.
Example
timestamp_col = next((c for c in model.columns if c.name == timestamp_field), None)
if timestamp_col:
normalized_type = timestamp_col.data_type.upper()
valid_ts_types = ['TIMESTAMP', 'TIMESTAMP_NTZ', 'TIMESTAMP_LTZ', 'TIMESTAMP_TZ', 'DATETIME', 'DATE']
if not any(t in normalized_type for t in valid_ts_types):
click.echo(f"{Fore.YELLOW}Warning: Timestamp field '{timestamp_field}' has type '{timestamp_col.data_type}' which may not be suitable{Style.RESET_ALL}")Related
Context
PR #5827 added dbt integration that requires a timestamp field for point-in-time correctness.
Problem
The code validates that the timestamp field exists in the model columns but doesn't validate that it's actually a timestamp type. This could lead to runtime errors or incorrect behavior.
Current Behavior
No type checking is performed.
Proposed Solution
Add validation that the timestamp field is one of the timestamp types:
Warn users if the field exists but has an incompatible type.
Example
Related