chore: add test of all supported BigQuery data types#17570
Conversation
This should be helpful in more thorough testing of the new compiler. Towards b/527509188
There was a problem hiding this comment.
Code Review
This pull request expands the nested_structs test dataset and schema to include a wide variety of data types (such as boolean, integers, floats, JSON, dates, timestamps, geography, and decimals) and edge cases. It also updates the nested_structs_pandas_df fixture to manually parse these types and adds a comprehensive test suite to verify their behavior. Feedback was provided regarding a potential compatibility issue with datetime.datetime.fromisoformat parsing the trailing 'Z' suffix in Python versions older than 3.11.
| timestamp_vals = [ | ||
| datetime.datetime.fromisoformat(get_val(row, "timestamp_col")) | ||
| if get_val(row, "timestamp_col") is not None | ||
| else None | ||
| for row in raw_rows | ||
| ] |
There was a problem hiding this comment.
In Python 3.10 and below, datetime.datetime.fromisoformat does not support the trailing Z suffix (which was introduced in Python 3.11). Since this library supports Python 3.9 and 3.10, parsing timestamp_col values ending in Z will raise a ValueError. Replacing Z with +00:00 ensures backward compatibility across all supported Python versions.
| timestamp_vals = [ | |
| datetime.datetime.fromisoformat(get_val(row, "timestamp_col")) | |
| if get_val(row, "timestamp_col") is not None | |
| else None | |
| for row in raw_rows | |
| ] | |
| timestamp_vals = [ | |
| datetime.datetime.fromisoformat(get_val(row, "timestamp_col").replace("Z", "+00:00")) | |
| if get_val(row, "timestamp_col") is not None | |
| else None | |
| for row in raw_rows | |
| ] |
This should be helpful in more thorough testing of the new compiler.
Towards b/527509188
🦕