feat: prepare client-side delegation and telemetry for pandas-gbq#17704
feat: prepare client-side delegation and telemetry for pandas-gbq#17704shuoweil wants to merge 8 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces query delegation support to the pandas-gbq package when retrieving dataframes via to_dataframe. It adds version helper utilities to check if delegation is supported, issues a deprecation warning if the core client fallback is used, and delegates the dataframe creation to pandas_gbq when available. The review feedback suggests making the pandas-gbq import and version checks more robust by catching general exceptions rather than just ImportError, using getattr to safely retrieve the package version, and ensuring that pyarrow is skipped in tests when not installed to prevent test failures.
| if self._installed_version is None: | ||
| try: | ||
| import pandas_gbq # type: ignore | ||
|
|
||
| self._installed_version = packaging.version.parse( | ||
| getattr(pandas_gbq, "__version__", "0.0.0") | ||
| ) | ||
| except ImportError: | ||
| self._installed_version = packaging.version.parse("0.0.0") |
There was a problem hiding this comment.
Catching only ImportError when importing pandas_gbq can lead to unhandled exceptions if the import fails due to other issues (e.g., compiled extension load failures, AttributeError, or TypeError from broken dependencies). Additionally, packaging.version.parse can raise InvalidVersion if the version string is malformed. Catching Exception ensures a robust fallback to version 0.0.0.
| if self._installed_version is None: | |
| try: | |
| import pandas_gbq # type: ignore | |
| self._installed_version = packaging.version.parse( | |
| getattr(pandas_gbq, "__version__", "0.0.0") | |
| ) | |
| except ImportError: | |
| self._installed_version = packaging.version.parse("0.0.0") | |
| if self._installed_version is None: | |
| try: | |
| import pandas_gbq # type: ignore | |
| self._installed_version = packaging.version.parse( | |
| getattr(pandas_gbq, "__version__", "0.0.0") | |
| ) | |
| except Exception: | |
| self._installed_version = packaging.version.parse("0.0.0") |
There was a problem hiding this comment.
I do not want to hide errors for users.
| if self._delegation_api_version is None: | ||
| try: | ||
| import pandas_gbq # type: ignore | ||
|
|
||
| self._delegation_api_version = getattr( | ||
| pandas_gbq, "_internal_delegation_api_version", 0 | ||
| ) | ||
| except ImportError: | ||
| self._delegation_api_version = 0 |
There was a problem hiding this comment.
Similar to installed_version, catching only ImportError when importing pandas_gbq can lead to unhandled exceptions if the import fails due to other issues. Catching Exception ensures a robust fallback to 0.
| if self._delegation_api_version is None: | |
| try: | |
| import pandas_gbq # type: ignore | |
| self._delegation_api_version = getattr( | |
| pandas_gbq, "_internal_delegation_api_version", 0 | |
| ) | |
| except ImportError: | |
| self._delegation_api_version = 0 | |
| if self._delegation_api_version is None: | |
| try: | |
| import pandas_gbq # type: ignore | |
| self._delegation_api_version = getattr( | |
| pandas_gbq, "_internal_delegation_api_version", 0 | |
| ) | |
| except Exception: | |
| self._delegation_api_version = 0 |
There was a problem hiding this comment.
I do not want to hide errors for users.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
Converting to draft until presubmits are green |
This pull request introduces the client-side framework in google-cloud-bigquery to support delegating high-throughput DataFrame conversions to the specialized pandas-gbq package.
These changes represent the first stage of consolidating duplicate data conversion logic from the core BigQuery clients into pandas-gbq, keeping the core clients lightweight while offering a high-performance backend.
Fixes #<526614511> 🦕