Skip to content

Commit a16413e

Browse files
Ace NassriTakashi Matsuo
Ace Nassri
and
Takashi Matsuo
authored
chore(Dockerfile): add Python 3.9 (GoogleCloudPlatform#4968)
* chore(Dockerfile): add Python 3.9 * Add py3.9 kokoro dir * fix typo * Add GPG keys * Add 3.9 to noxfiles * Update composer dep to avoid deprecation spam * fix(storage): add py-3.9 specific key * update psycopg2-binary, only run test in py-3.9 build * add libmemcached-dev to the Dockerfile * disable appengine standard test in py-3.9 build * disable py-3.9 build for appengine cloud_debugger * skip py-3.9 build for composer/workflows * skip tests with pyarrow for py-3.9 build * avoid ReferenceError in iot builds * skip some tests due to pip error * add a temporary statement for debugging * fix lint * use correct constant * disable 2.7 builds * disable builds due to pip conflict The conflict is between google-cloud-monitoring==2.0.0 and opencensus-ext-stackdriver. * remove temporary debugging statement * really skip py-3.9 build for pubsub/streaming-analytics * copyright year fix * fix(storage): explicitly use the test project for the test bucket * fix(storage): use correct cloud project * fix: disable py-3.9 builds - appengine/standard_python3/bigquery - data-science-onramp/data-ingestion * disable py-3.9 build - dataflow/encryption-keys - dataflow/flex-templates/streaming_beam * disable type hint checks Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
1 parent 9b43aa7 commit a16413e

File tree

44 files changed

+653
-33
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+653
-33
lines changed

‎.kokoro/docker/Dockerfile‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ RUN apt-get update \
4545
libffi-dev \
4646
liblzma-dev \
4747
libmagickwand-dev \
48+
libmemcached-dev \
4849
libpython3-dev \
4950
libreadline-dev \
5051
libsnappy-dev \
@@ -103,7 +104,7 @@ RUN set -ex \
103104
&& export GNUPGHOME="$(mktemp -d)" \
104105
&& echo "disable-ipv6" >> "${GNUPGHOME}/dirmngr.conf" \
105106
&& /tmp/fetch_gpg_keys.sh \
106-
&& for PYTHON_VERSION in 2.7.18 3.6.10 3.7.7 3.8.3; do \
107+
&& for PYTHON_VERSION in 2.7.18 3.6.10 3.7.7 3.8.3 3.9.0; do \
107108
wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz" \
108109
&& wget --no-check-certificate -O python-${PYTHON_VERSION}.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc" \
109110
&& gpg --batch --verify python-${PYTHON_VERSION}.tar.xz.asc python-${PYTHON_VERSION}.tar.xz \

‎.kokoro/docker/fetch_gpg_keys.sh‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,10 @@ retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
4444
retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
4545
0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D
4646

47-
# 3.8.0 (Łukasz Langa)
47+
# 3.8.0, 3.9.0 (Łukasz Langa)
4848
retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
4949
E3FF2839C048B25C084DEBE9B26995E310250568
50+
51+
# 3.10.x and 3.11.x (Pablo Galindo Salgado)
52+
retry 3 gpg --keyserver ha.pool.sks-keyservers.net --recv-keys \
53+
A035C8C19219BA821ECEA86B64E628F8D684696D

‎.kokoro/python3.9/common.cfg‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Copyright 2019 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Format: //devtools/kokoro/config/proto/build.proto
16+
17+
timeout_mins: 300
18+
19+
# Configure the docker image for kokoro-trampoline.
20+
env_vars: {
21+
key: "TRAMPOLINE_IMAGE"
22+
value: "gcr.io/cloud-devrel-kokoro-resources/python-samples-testing-docker"
23+
}
24+
25+
# Download trampoline resources.
26+
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"
27+
28+
# Use the trampoline script to run in docker.
29+
build_file: "python-docs-samples/.kokoro/trampoline_v2.sh"
30+
31+
# Download secrets from Cloud Storage.
32+
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/python-docs-samples"
33+
34+
# Copy results for Resultstore
35+
action {
36+
define_artifacts {
37+
regex: "**/*sponge_log.xml"
38+
}
39+
}
40+
41+
# Specify which tests to run
42+
env_vars: {
43+
key: "RUN_TESTS_SESSION"
44+
value: "py-3.9"
45+
}
46+
47+
# Declare build specific Cloud project. It still uses the common one,
48+
# but we'll update the value once we have more Cloud projects.
49+
env_vars: {
50+
key: "BUILD_SPECIFIC_GCLOUD_PROJECT"
51+
value: "python-docs-samples-tests-py39"
52+
}
53+
54+
# Number of test workers.
55+
env_vars: {
56+
key: "NUM_TEST_WORKERS"
57+
value: "10"
58+
}
59+
60+
env_vars: {
61+
key: "TRAMPOLINE_DOCKERFILE"
62+
value: ".kokoro/docker/Dockerfile"
63+
}

‎.kokoro/python3.9/continuous.cfg‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Copyright 2020 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Format: //devtools/kokoro/config/proto/build.proto
16+
17+
# Tell the trampoline which build file to use.
18+
env_vars: {
19+
key: "TRAMPOLINE_BUILD_FILE"
20+
value: ".kokoro/tests/run_tests_diff_head.sh"
21+
}

‎.kokoro/python3.9/periodic.cfg‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Copyright 2020 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Format: //devtools/kokoro/config/proto/build.proto
16+
17+
# Tell the trampoline which build file to use.
18+
env_vars: {
19+
key: "TRAMPOLINE_BUILD_FILE"
20+
value: ".kokoro/tests/run_tests.sh"
21+
}
22+
23+
env_vars: {
24+
key: "REPORT_TO_BUILD_COP_BOT"
25+
value: "true"
26+
}
27+
28+
# Tell Trampoline to upload the Docker image after successfull build.
29+
env_vars: {
30+
key: "TRAMPOLINE_IMAGE_UPLOAD"
31+
value: "true"
32+
}

‎.kokoro/python3.9/presubmit.cfg‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Copyright 2019 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Format: //devtools/kokoro/config/proto/build.proto
16+
17+
# Tell the trampoline which build file to use.
18+
env_vars: {
19+
key: "TRAMPOLINE_BUILD_FILE"
20+
value: ".kokoro/tests/run_tests_diff_master.sh"
21+
}

‎appengine/flexible/memcache/noxfile.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222

2323
# DO NOT EDIT - automatically generated.
2424
# All versions used to tested samples.
25-
ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8"]
25+
ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8", "3.9"]
2626

2727
# Any default versions that should be ignored.
2828
IGNORED_VERSIONS = ["2.7", "3.8"]

‎appengine/standard/noxfile-template.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,10 +81,10 @@ def get_pytest_env_vars():
8181

8282
# DO NOT EDIT - automatically generated.
8383
# All versions used to tested samples.
84-
ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8"]
84+
ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8", "3.9"]
8585

8686
# Any default versions that should be ignored.
87-
IGNORED_VERSIONS = ["3.6", "3.7", "3.8"]
87+
IGNORED_VERSIONS = ["3.6", "3.7", "3.8", "3.9"]
8888

8989
TESTED_VERSIONS = sorted([v for v in ALL_VERSIONS if v not in IGNORED_VERSIONS])
9090

‎appengine/standard_python3/bigquery/noxfile_config.py‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Copyright 2021 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Default TEST_CONFIG_OVERRIDE for python repos.
16+
17+
# You can copy this file into your directory, then it will be inported from
18+
# the noxfile.py.
19+
20+
# The source of truth:
21+
# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/noxfile_config.py
22+
23+
TEST_CONFIG_OVERRIDE = {
24+
# You can opt out from the test for specific Python versions.
25+
# There's no google-cloud-bigquery package for Python 3.9.
26+
"ignored_versions": ["2.7", "3.9"],
27+
# Old samples are opted out of enforcing Python type hints
28+
# All new samples should feature them
29+
"enforce_type_hints": False,
30+
# An envvar key for determining the project id to use. Change it
31+
# to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
32+
# build specific Cloud project. You can also use your own string
33+
# to use your own Cloud project.
34+
"gcloud_project_env": "GOOGLE_CLOUD_PROJECT",
35+
# 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
36+
# A dictionary you want to inject into your test. Don't put any
37+
# secrets here. These values will override predefined values.
38+
"envs": {},
39+
}

‎appengine/standard_python3/cloud_debugger/noxfile_config.py‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Copyright 2021 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Default TEST_CONFIG_OVERRIDE for python repos.
16+
17+
# You can copy this file into your directory, then it will be inported from
18+
# the noxfile.py.
19+
20+
# The source of truth:
21+
# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/noxfile_config.py
22+
23+
TEST_CONFIG_OVERRIDE = {
24+
# You can opt out from the test for specific Python versions.
25+
# google-python-cloud-debugger doesn't support Python 3.9.
26+
# https://github.com/GoogleCloudPlatform/cloud-debug-python/issues/22
27+
"ignored_versions": ["2.7", "3.9"],
28+
# Old samples are opted out of enforcing Python type hints
29+
# All new samples should feature them
30+
"enforce_type_hints": False,
31+
# An envvar key for determining the project id to use. Change it
32+
# to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
33+
# build specific Cloud project. You can also use your own string
34+
# to use your own Cloud project.
35+
"gcloud_project_env": "GOOGLE_CLOUD_PROJECT",
36+
# 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
37+
# A dictionary you want to inject into your test. Don't put any
38+
# secrets here. These values will override predefined values.
39+
"envs": {},
40+
}

0 commit comments

Comments
 (0)