Skip to content

BigQuery: Use configurable bucket name for GCS samples data in systems tests. #8783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tswast merged 3 commits into from
Jul 29, 2019
Merged

BigQuery: Use configurable bucket name for GCS samples data in systems tests. #8783

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
cdc063b
Use configurable bucket name for GCS samples data in systems tests.
tswast Jul 25, 2019
d7eca15
Merge remote-tracking branch 'upstream/master' into issue8550-bq-vpc-…
tswast Jul 26, 2019
cb46cdf
Blacken
tswast Jul 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion bigquery/tests/system.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,11 @@
),
]

# The VPC-SC team maintains a mirror of the GCS bucket used for code
# samples. The public bucket crosses the configured security boundary.
# See: https://github.com/googleapis/google-cloud-python/issues/8550
SAMPLES_BUCKET = os.environ.get("GCLOUD_TEST_SAMPLES_BUCKET", "cloud-samples-data")

@tseaver tseaver Jul 25, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update noxfile.py to set this value in the environment?

@tswast tswast Jul 25, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the default of cloud-samples-data is fine for our purposes. The VPC-SC team has a separate Kokoro (or maybe something similar?) config where they'll set the necessary environment variable.


retry_storage_errors = RetryErrors(
(TooManyRequests, InternalServerError, ServiceUnavailable)
)
Expand Down Expand Up @@ -1877,7 +1882,9 @@ def test_create_routine(self):
language="JAVASCRIPT",
type_="SCALAR_FUNCTION",
return_type=float64_type,
imported_libraries=["gs://cloud-samples-data/bigquery/udfs/max-value.js"],
imported_libraries=[
"gs://{}/bigquery/udfs/max-value.js".format(SAMPLES_BUCKET)
],
)
routine.arguments = [
bigquery.RoutineArgument(
Expand Down