chore: migrate code from googleapis/python-db-dtypes-pandas (GoogleCloudPlatform#13776)

Linchin · tswast · gcf-owl-bot[bot] · web-flow · commit 92d6169f0aa8 · 2026-02-02T22:01:47.000-08:00
* feat: add `time` and `date` dtypes Migrated from BigQuery googleapis/python-bigquery#972 * chore: add license headers to __init__.py files * chore: fail samples nox session if python version is missing (GoogleCloudPlatform#17) * chore(python): Add kokoro configs for python 3.10 samples testing (GoogleCloudPlatform#26) * feat: rename dbtime and dbdate dtypes to avoid future conflicts with pandas (GoogleCloudPlatform#32) * docs: add how-to guide and include API reference (GoogleCloudPlatform#33) * docs: add how-to guide and include API reference * fix indentation * fix types in sample * fix types in sample * chore(samples): Add check for tests in directory (GoogleCloudPlatform#54) Source-Link: googleapis/synthtool@52aef91 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:36a95b8f494e4674dc9eee9af98961293b51b86b3649942aac800ae6c1f796d4 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(python): Noxfile recognizes that tests can live in a folder (GoogleCloudPlatform#58) Source-Link: googleapis/synthtool@4760d8d Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f0e4b51deef56bed74d3e2359c583fc104a8d6367da3984fc5c66938db738828 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update all dependencies (GoogleCloudPlatform#68) * chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Update setup.py Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <swast@google.com> * chore: Adding support for pytest-xdist and pytest-parallel (GoogleCloudPlatform#76) Source-Link: googleapis/synthtool@82f5cb2 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:5d8da01438ece4021d135433f2cf3227aa39ef0eaccc941d62aa35e6902832ae Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update dependency pytest to v7.1.0 (GoogleCloudPlatform#80) * chore(deps): update dependency pytest to v7.1.1 (GoogleCloudPlatform#83) * chore(python): use black==22.3.0 (GoogleCloudPlatform#96) Source-Link: googleapis/synthtool@6fab84a Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:7cffbc10910c3ab1b852c05114a08d374c195a81cdec1d4a67a1d129331d0bfe * chore(python): add nox session to sort python imports (GoogleCloudPlatform#102) Source-Link: googleapis/synthtool@1b71c10 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:00c9d764fd1cd56265f12a5ef4b99a0c9e87cf261018099141e2ca5158890416 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update dependency pytest to v7.1.2 (GoogleCloudPlatform#105) * fix: require python 3.7+ (GoogleCloudPlatform#125) * chore(python): drop python 3.6 Source-Link: googleapis/synthtool@4f89b13 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e7bb19d47c13839fe8c147e50e02e8b6cf5da8edd1af8b82208cd6f66cc2829c * require python 3.7+ in setup.py * remove python 3.6 sample configs * update product documentation in .repo-metadata.json * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * exclude templated readme * require python 3.7+ in setup.py * remove python 3.6 from noxfile * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * update README Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com> * chore(deps): update dependency pytest to v7.1.3 (GoogleCloudPlatform#144) * chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com> * chore: detect samples tests in nested directories (GoogleCloudPlatform#146) Source-Link: googleapis/synthtool@50db768 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e09366bdf0fd9c8976592988390b24d53583dd9f002d476934da43725adbb978 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update dependency pytest to v7.2.0 (GoogleCloudPlatform#151) * chore(python): drop flake8-import-order in samples noxfile (GoogleCloudPlatform#156) Source-Link: googleapis/synthtool@6ed3a83 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:3abfa0f1886adaf0b83f07cb117b24a639ea1cb9cffe56d43280b977033563eb Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(python): add support for python 3.11 (GoogleCloudPlatform#168) Source-Link: googleapis/synthtool@7197a00 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:c43f1d918bcf817d337aa29ff833439494a158a0831508fda4ec75dc4c0d0320 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update dependency pytest to v7.2.1 (GoogleCloudPlatform#170) * chore(deps): update dependency pytest to v7.2.1 * use python 3.11 in prerelease github action workflow * use python 3.11 in compliance github action workflow --------- Co-authored-by: Anthonios Partheniou <partheniou@google.com> * chore(deps): update dependency pytest to v7.2.2 (GoogleCloudPlatform#174) * chore(deps): update dependency pytest to v7.3.1 (GoogleCloudPlatform#184) * chore(deps): update dependency pytest to v7.3.2 (GoogleCloudPlatform#188) Co-authored-by: Anthonios Partheniou <partheniou@google.com> * chore(deps): update dependency pytest to v7.4.0 (GoogleCloudPlatform#191) Co-authored-by: Anthonios Partheniou <partheniou@google.com> * chore(deps): update all dependencies (GoogleCloudPlatform#209) * chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update all dependencies (GoogleCloudPlatform#221) * chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: Add support for Python 3.12 (GoogleCloudPlatform#223) * chore(python): Add Python 3.12 Source-Link: googleapis/synthtool@af16e6d Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:bacc3af03bff793a03add584537b36b5644342931ad989e3ba1171d3bd5399f5 * add constraints file for python 3.12 * remove constraints file for python 3.6 * Add trove classifiers for python 3.11 / 3.12 * update prerelease and compliance workflows to use python 3.12 --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com> * chore(deps): update dependency pytest to v7.4.4 (GoogleCloudPlatform#235) Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com> * chore(deps): update dependency pytest to v8 (GoogleCloudPlatform#239) * chore(deps): update dependency pytest to v8 * pin pytest == 7.4.4 for python 3.7 --------- Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com> * chore(deps): update dependency pytest to v8.0.1 (GoogleCloudPlatform#243) * chore(deps): update dependency pytest to v8.0.1 * pin pytest version for python==3.7 --------- Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com> * chore(deps): update dependency pytest to v8.0.2 (GoogleCloudPlatform#247) * chore(deps): update dependency pytest to v8.0.2 * pin pytest==7.4.4 for python 3.7 --------- Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com> * chore(deps): update dependency pytest to v8.1.1 (GoogleCloudPlatform#251) * chore(deps): update dependency pytest to v8.1.1 * pin pytest==7.4.4 for python 3.7 --------- Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com> * testing: use arbitrary equality for pinned pytest version (GoogleCloudPlatform#262) * chore: format code files with nox (GoogleCloudPlatform#281) * chore(deps): update all dependencies (GoogleCloudPlatform#290) * chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * build: use multiScm for Kokoro release builds (GoogleCloudPlatform#294) Source-Link: googleapis/synthtool@0da1658 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:5cddfe2fb5019bbf78335bc55f15bc13e18354a56b3ff46e1834f8e540807f05 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * deps!: Drop support for Python 3.7 and 3.8 (AI Experiment) (GoogleCloudPlatform#337) * feat: Drop support for Python 3.7 and 3.8 Removes support for Python 3.7 and 3.8, establishing Python 3.9 as the new minimum supported version. This change involves: - Updating `python_requires` and classifiers in `setup.py`. - Modifying Python versions in `noxfile.py` (default, unit tests, system tests) and ensuring constraint file logic remains correct. - Updating the GitHub Actions workflow (`unittest.yml`) matrix, runner, and coverage job version. - Deleting constraint files for Python 3.7 and 3.8 (`testing/constraints-3.7.txt`, `testing/constraints-3.8.txt`). - Removing Kokoro sample configuration directories (`.kokoro/samples/python3.7/`, `.kokoro/samples/python3.8/`). - Updating supported version mentions in `README.rst`. - Removing 3.7 and 3.8 from the `ALL_VERSIONS` list in `samples/snippets/noxfile.py`. * Updates python version in lint.yml * Updates owlbot, removing reference to 3.8 * Updates CONTRIBUTING.rst * updates pytest warnings * Removes test_samples-impl ref to older virtualenv package * Removes references to pandas older than 1.5.0 * Removes pandas older than 1.5 and misc changes * updates pandas in setup.py * more updates related to pandas * still broken * Updates FutureWarning tests to account for unittest coverage * Updates json array type tests to account for unittest coverage * updates python version checks to ensure coverage * update json test for unittest coverage * Update pandas_backports unittests to ensure coverage * Updates per review comments * moves class from version specific compliance file to generic file * Removes weird cut and paste error * fix linting errors * updates import statement to ensure import of JSONArrowType * Revise required github status checks * update linting * temporarily marking a class as no cover * more updates * marked several snippets as pragma no cover * updates linting * Updates constraints and setup.py * migrates class from one time compliance file to another * updating pyarrow version * Updates linting * removes determine all and module reload tests * updates re: ndarrybackedextensionarray * testing blacken as part of owlbot processing using 3.8 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * updates blacken to 3.10 * update python version in lint.yml * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * updates owlbot.py * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * updates owlbot.py * testing lint.yml * testing linting issue * testing linting issue * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * updates lint_setup session * Update noxfile.py --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * test: updates python versions and sets owlbot excludes (GoogleCloudPlatform#367) * chore(deps): update all dependencies * updates to prevent changes to docs and lint.ymls * updates lint python version to 3.10 * remove comments, update lint python version * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update dependency pytest to v8.4.1 (GoogleCloudPlatform#368) * chore(deps): update all dependencies * Update docs.yml * Update lint.yml --------- Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com> * chore(deps): update all dependencies (GoogleCloudPlatform#373) * feat: Add support for Python 3.14 (GoogleCloudPlatform#380) This PR adds Python 3.14 as a supported version, and fixed tests for the new version This PR also adds a dependency bound to pandas, as the [upcoming 3.0.0 version is incompatible](https://github.com/googleapis/python-db-dtypes-pandas/issues/381) * correct header * add noxconfig.py * delete noxfile.py * add * lint --------- Co-authored-by: Tim Swast <swast@google.com> Co-authored-by: gcf-owl-bot[bot] <78513119+gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: WhiteSource Renovate <bot@renovateapp.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com> Co-authored-by: Chelsea Lin <124939984+chelsea-lin@users.noreply.github.com> Co-authored-by: Chalmer Lowe <chalmer.lowe@gmail.com> Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Daniel Sanche <sanche@google.com>
diff --git a/bigquery/python-db-dtypes-pandas/__init__.py b/bigquery/python-db-dtypes-pandas/__init__.py
@@ -0,0 +1,13 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
diff --git a/bigquery/python-db-dtypes-pandas/pytest.ini b/bigquery/python-db-dtypes-pandas/pytest.ini
diff --git a/bigquery/python-db-dtypes-pandas/snippets/__init__.py b/bigquery/python-db-dtypes-pandas/snippets/__init__.py
@@ -0,0 +1,13 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
diff --git a/bigquery/python-db-dtypes-pandas/snippets/noxconfig.py b/bigquery/python-db-dtypes-pandas/snippets/noxconfig.py
@@ -0,0 +1,42 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Default TEST_CONFIG_OVERRIDE for python repos.
+
+# You can copy this file into your directory, then it will be imported from
+# the noxfile.py.
+
+# The source of truth:
+# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/noxfile_config.py
+
+TEST_CONFIG_OVERRIDE = {
+    # You can opt out from the test for specific Python versions.
+    "ignored_versions": ["2.7", "3.7", "3.8"],
+    # Old samples are opted out of enforcing Python type hints
+    # All new samples should feature them
+    "enforce_type_hints": True,
+    # An envvar key for determining the project id to use. Change it
+    # to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
+    # build specific Cloud project. You can also use your own string
+    # to use your own Cloud project.
+    "gcloud_project_env": "GOOGLE_CLOUD_PROJECT",
+    # 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
+    # If you need to use a specific version of pip,
+    # change pip_version_override to the string representation
+    # of the version number, for example, "20.2.4"
+    "pip_version_override": None,
+    # A dictionary you want to inject into your test. Don't put any
+    # secrets here. These values will override predefined values.
+    "envs": {},
+}
diff --git a/bigquery/python-db-dtypes-pandas/snippets/pandas_date_and_time.py b/bigquery/python-db-dtypes-pandas/snippets/pandas_date_and_time.py
@@ -0,0 +1,79 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+def pandas_date_and_time():
+    # [START bigquery_pandas_date_create]
+
+    import datetime
+
+    import pandas as pd
+
+    import db_dtypes  # noqa import to register dtypes
+
+    dates = pd.Series([datetime.date(2021, 9, 17), "2021-9-18"], dtype="dbdate")
+
+    # [END bigquery_pandas_date_create]
+    # [START bigquery_pandas_date_as_datetime]
+
+    datetimes = dates.astype("datetime64")
+
+    # [END bigquery_pandas_date_as_datetime]
+    # [START bigquery_pandas_date_sub]
+
+    dates2 = pd.Series(["2021-1-1", "2021-1-2"], dtype="dbdate")
+    diffs = dates - dates2
+
+    # [END bigquery_pandas_date_sub]
+    # [START bigquery_pandas_date_add_offset]
+
+    do = pd.DateOffset(days=1)
+    after = dates + do
+    before = dates - do
+
+    # [END bigquery_pandas_date_add_offset]
+    # [START bigquery_pandas_time_create]
+
+    times = pd.Series([datetime.time(1, 2, 3, 456789), "12:00:00.6"], dtype="dbtime")
+
+    # [END bigquery_pandas_time_create]
+    # [START bigquery_pandas_time_as_timedelta]
+
+    timedeltas = times.astype("timedelta64")
+
+    # [END bigquery_pandas_time_as_timedelta]
+
+    # Combine datetime64 and timedelta64 to confirm adding dates and times are
+    # equivalent.
+    combined0 = datetimes + timedeltas
+
+    # [START bigquery_pandas_combine_date_time]
+
+    combined = dates + times
+
+    # [END bigquery_pandas_combine_date_time]
+
+    return (
+        dates,
+        datetimes,
+        dates2,
+        diffs,
+        do,
+        after,
+        before,
+        times,
+        timedeltas,
+        combined,
+        combined0,
+    )
diff --git a/bigquery/python-db-dtypes-pandas/snippets/pandas_date_and_time_test.py b/bigquery/python-db-dtypes-pandas/snippets/pandas_date_and_time_test.py
@@ -0,0 +1,60 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import datetime
+
+import numpy as np
+from pandas import Timestamp
+
+
+def test_pandas_date_and_time():
+    from .pandas_date_and_time import pandas_date_and_time
+
+    (
+        dates,
+        _,
+        dates2,
+        diffs,
+        do,
+        after,
+        before,
+        times,
+        _,
+        combined,
+        combined0,
+    ) = pandas_date_and_time()
+
+    assert str(dates.dtype) == "dbdate"
+    assert list(dates) == [datetime.date(2021, 9, 17), datetime.date(2021, 9, 18)]
+
+    assert np.array_equal(
+        diffs,
+        dates.astype("datetime64") - dates2.astype("datetime64"),
+    )
+
+    assert np.array_equal(after, dates.astype("object") + do)
+    assert np.array_equal(before, dates.astype("object") - do)
+
+    assert str(times.dtype) == "dbtime"
+    assert list(times) == [
+        datetime.time(1, 2, 3, 456789),
+        datetime.time(12, 0, 0, 600000),
+    ]
+
+    for c in combined0, combined:
+        assert str(c.dtype) == "datetime64[ns]"
+        assert list(c) == [
+            Timestamp("2021-09-17 01:02:03.456789"),
+            Timestamp("2021-09-18 12:00:00.600000"),
+        ]
diff --git a/bigquery/python-db-dtypes-pandas/snippets/requirements-test.txt b/bigquery/python-db-dtypes-pandas/snippets/requirements-test.txt
@@ -0,0 +1 @@
+pytest==8.4.2
diff --git a/bigquery/python-db-dtypes-pandas/snippets/requirements.txt b/bigquery/python-db-dtypes-pandas/snippets/requirements.txt
@@ -0,0 +1,4 @@
+db-dtypes
+numpy
+pandas
+pyarrow

-Original file line number
+Diff line change
@@ @@ -0,0 +1,4 @@ @@
 +db-dtypes
 +numpy
 +pandas
 +pyarrow