Skip to content

test: add dual-environment validation tests for pandas-gbq delegation - #18018

Open
shuoweil wants to merge 9 commits into
mainfrom
shuowei-gbq-storage-mvp-validation
Open

test: add dual-environment validation tests for pandas-gbq delegation#18018
shuoweil wants to merge 9 commits into
mainfrom
shuowei-gbq-storage-mvp-validation

Conversation

@shuoweil

@shuoweil shuoweil commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Adds end-to-end unit test validation for ReadRowsPage.to_arrow() delegation to pandas-gbq and fallback behavior when pandas-gbq is uninstalled.

Changes

  • Added packages/google-cloud-bigquery-storage/tests/unit/test_reader_pandas_gbq.py
  • Tests real protobuf decoding and Arrow RecordBatch assembly across multiple BigQuery data types (ints, floats, strings, decimals, dates, UTC timestamps).
  • Tests empty batch response handling.
  • Tests multi-page stream concatenation.
  • Tests safe fallback to legacy in-tree parser when pandas-gbq is absent.

Fixes #< 540939659 > 🦕

@shuoweil
shuoweil requested review from a team as code owners August 6, 2026 20:29
@shuoweil
shuoweil requested review from tswast and removed request for a team August 6, 2026 20:29
@shuoweil
shuoweil marked this pull request as draft August 6, 2026 20:29
@shuoweil
shuoweil removed the request for review from tswast August 6, 2026 20:29

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new test suite test_reader_pandas_gbq.py to verify Arrow serialization and deserialization behaviors, including delegation to pandas_gbq and fallback paths when it is not installed. The review feedback points out that importing pandas_gbq.arrow at the top level will cause test discovery to fail in environments where pandas-gbq is missing. It suggests dynamically importing the optional dependency using pytest.importorskip within the relevant tests to ensure robust test execution and proper path isolation.

import decimal
from unittest import mock

import pandas_gbq.arrow

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Importing pandas_gbq.arrow at the top level will cause the entire test module to fail to load in environments where pandas-gbq is not installed. Since this file contains tests specifically designed to verify the fallback behavior when pandas-gbq is not installed (e.g., test_read_rows_page_to_arrow_falls_back_when_pandas_gbq_uninstalled), we should avoid top-level imports of optional dependencies. Instead, import them dynamically within the specific tests that require them.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

shuoweil and others added 6 commits August 6, 2026 13:54
…pandas_gbq.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…pandas_gbq.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…pandas_gbq.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…pandas_gbq.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@shuoweil
shuoweil marked this pull request as ready for review August 6, 2026 21:15
@shuoweil
shuoweil requested review from sycai and tswast August 6, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant