Skip to content

Commit 235417b

Browse files
yoshi-automationbusunkim96
yoshi-automation
and
busunkim96
authored
test: use constraints files to test lower bounds (#41)
* changes without context autosynth cannot find the source of changes triggered by earlier changes in this repository, or by version upgrades to tools such as linters. * chore(python): fix column sizing issue in docs Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Thu Jan 7 11:58:32 2021 -0700 Source-Repo: googleapis/synthtool Source-Sha: f15b57ccfd71106c2299e9b89835fe6e55015662 Source-Link: googleapis/synthtool@f15b57c * chore(python): use 'http' in LICENSE Co-authored-by: Tim Swast <swast@google.com> Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Thu Jan 7 13:05:12 2021 -0700 Source-Repo: googleapis/synthtool Source-Sha: 41a4e56982620d3edcf110d76f4fcdfdec471ac8 Source-Link: googleapis/synthtool@41a4e56 * chore(python): skip docfx in main presubmit * chore(python): skip docfx in main presubmit * chore: add missing quotation mark Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Mon Jan 11 09:43:06 2021 -0700 Source-Repo: googleapis/synthtool Source-Sha: 16ec872dd898d7de6e1822badfac32484b5d9031 Source-Link: googleapis/synthtool@16ec872 * build(python): make `NOX_SESSION` optional I added this accidentally in #889. `NOX_SESSION` should be passed down if it is set but not marked required. Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Tue Jan 19 09:38:04 2021 -0700 Source-Repo: googleapis/synthtool Source-Sha: ba960d730416fe05c50547e975ce79fcee52c671 Source-Link: googleapis/synthtool@ba960d7 * chore: add noxfile parameters for extra dependencies Also, add tests for some noxfile parameters for assurance that the template generates valid Python. Co-authored-by: Jeffrey Rennie <rennie@google.com> Source-Author: Tim Swast <swast@google.com> Source-Date: Tue Jan 26 12:26:57 2021 -0600 Source-Repo: googleapis/synthtool Source-Sha: 778d8beae28d6d87eb01fdc839a4b4d966ed2ebe Source-Link: googleapis/synthtool@778d8be * build: migrate to flakybot Source-Author: Justin Beckwith <justin.beckwith@gmail.com> Source-Date: Thu Jan 28 22:22:38 2021 -0800 Source-Repo: googleapis/synthtool Source-Sha: d1bb9173100f62c0cfc8f3138b62241e7f47ca6a Source-Link: googleapis/synthtool@d1bb917 * chore(python): include py.typed files in release A py.typed file must be included in the released package for it to be considered typed by type checkers. https://www.python.org/dev/peps/pep-0561/#packaging-type-information. See googleapis/python-secret-manager#79 Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Fri Feb 5 17:32:06 2021 -0700 Source-Repo: googleapis/synthtool Source-Sha: 33366574ffb9e11737b3547eb6f020ecae0536e8 Source-Link: googleapis/synthtool@3336657 * docs: update python contributing guide Adds details about blacken, updates version for system tests, and shows how to pass through pytest arguments. Source-Author: Chris Cotter <cjcotter@google.com> Source-Date: Mon Feb 8 17:13:36 2021 -0500 Source-Repo: googleapis/synthtool Source-Sha: 4679e7e415221f03ff2a71e3ffad75b9ec41d87e Source-Link: googleapis/synthtool@4679e7e * build(python): enable flakybot on library unit and system tests Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Wed Feb 17 14:10:46 2021 -0700 Source-Repo: googleapis/synthtool Source-Sha: d17674372e27fb8f23013935e794aa37502071aa Source-Link: googleapis/synthtool@d176743 * test: install pyopenssl for mtls testing Source-Author: arithmetic1728 <58957152+arithmetic1728@users.noreply.github.com> Source-Date: Tue Mar 2 12:27:56 2021 -0800 Source-Repo: googleapis/synthtool Source-Sha: 0780323da96d5a53925fe0547757181fe76e8f1e Source-Link: googleapis/synthtool@0780323 * chore: add pre-commit-config to renovate ignore paths Disable renovate PRs on the .pre-commit-config.yaml which is templated from synthtool. https://docs.renovatebot.com/configuration-options/#ignorepaths Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Mon Mar 15 09:05:39 2021 -0600 Source-Repo: googleapis/synthtool Source-Sha: 2c54c473779ea731128cea61a3a6c975a08a5378 Source-Link: googleapis/synthtool@2c54c47 * build(python): fail nox sessions if a python version is missing Nox's default behavior is to quietly skip if a python interpreter is missing. https://nox.thea.codes/en/stable/usage.html#failing-sessions-when-the-interpreter-is-missing Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Tue Mar 16 13:38:02 2021 -0600 Source-Repo: googleapis/synthtool Source-Sha: eda422b90c3dde4a872a13e6b78a8f802c40d0db Source-Link: googleapis/synthtool@eda422b * chore(python): add kokoro configs for periodic builds against head This change should be non-destructive. Note for library repo maintainers: After applying this change, you can easily add (or change) periodic builds against head by adding config files in google3. See python-pubsub repo for example. Source-Author: Takashi Matsuo <tmatsuo@google.com> Source-Date: Fri Mar 19 11:17:59 2021 -0700 Source-Repo: googleapis/synthtool Source-Sha: 79c8dd7ee768292f933012d3a69a5b4676404cda Source-Link: googleapis/synthtool@79c8dd7 * chore(deps): update precommit hook pycqa/flake8 to v3.9.0 [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [pycqa/flake8](https://gitlab.com/pycqa/flake8) | repository | minor | `3.8.4` -> `3.9.0` | --- ### Release Notes <details> <summary>pycqa/flake8</summary> ### [`v3.9.0`](https://gitlab.com/pycqa/flake8/compare/3.8.4...3.9.0) [Compare Source](https://gitlab.com/pycqa/flake8/compare/3.8.4...3.9.0) </details> --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/synthtool). Source-Author: WhiteSource Renovate <bot@renovateapp.com> Source-Date: Tue Mar 23 17:38:03 2021 +0100 Source-Repo: googleapis/synthtool Source-Sha: f5c5904fb0c6aa3b3730eadf4e5a4485afc65726 Source-Link: googleapis/synthtool@f5c5904 * test(python): use constraints files to check dependency lower bounds Use a constraints file when installing dependencies for system and unit tests nox sessions. https://pip.pypa.io/en/stable/user_guide/#constraints-files > Constraints files are requirements files that **only control which version of a requirement is installed, not whether it is installed or not**. Their syntax and contents is nearly identical to Requirements Files. There is one key difference: Including a package in a constraints file does not trigger installation of the package. ``` testing ├── constraints-3.10.txt ├── constraints-3.11.txt ├── constraints-3.6.txt ├── constraints-3.7.txt ├── constraints-3.8.txt └── constraints-3.9.txt ``` Going forward, one constraints file (currently 3.6) will be populated with every library requirement and extra listed in the `setup.py`. The constraints file will pin each requirement to the lower bound. This ensures that library maintainers will see test failures if they forget to update a lower bound on a dependency. See googleapis/python-bigquery#263 for an example Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Source-Date: Tue Mar 23 10:52:02 2021 -0600 Source-Repo: googleapis/synthtool Source-Sha: 86ed43d4f56e6404d068e62e497029018879c771 Source-Link: googleapis/synthtool@86ed43d * chore: add constraints files * chore: fix docstring * chore: exclude readme from gen Co-authored-by: Bu Sun Kim <busunkim@google.com>
1 parent 66140da commit 235417b

File tree

36 files changed

+832
-439
lines changed

36 files changed

+832
-439
lines changed

‎packages/google-cloud-memcache/.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,8 +50,10 @@ docs.metadata
5050

5151
# Virtual environment
5252
env/
53+
54+
# Test logs
5355
coverage.xml
54-
sponge_log.xml
56+
*sponge_log.xml
5557

5658
# System test environment variables.
5759
system_tests/local_test_setup

‎packages/google-cloud-memcache/.kokoro/build.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,16 @@ python3 -m pip uninstall --yes --quiet nox-automation
4040
python3 -m pip install --upgrade --quiet nox
4141
python3 -m nox --version
4242

43+
# If this is a continuous build, send the test log to the FlakyBot.
44+
# See https://github.com/googleapis/repo-automation-bots/tree/master/packages/flakybot.
45+
if [[ $KOKORO_BUILD_ARTIFACTS_SUBDIR = *"continuous"* ]]; then
46+
cleanup() {
47+
chmod +x $KOKORO_GFILE_DIR/linux_amd64/flakybot
48+
$KOKORO_GFILE_DIR/linux_amd64/flakybot
49+
}
50+
trap cleanup EXIT HUP
51+
fi
52+
4353
# If NOX_SESSION is set, it only runs the specified session,
4454
# otherwise run all the sessions.
4555
if [[ -n "${NOX_SESSION:-}" ]]; then

‎packages/google-cloud-memcache/.kokoro/samples/python3.6/periodic-head.cfg‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Format: //devtools/kokoro/config/proto/build.proto
2+
3+
env_vars: {
4+
key: "INSTALL_LIBRARY_FROM_SOURCE"
5+
value: "True"
6+
}
7+
8+
env_vars: {
9+
key: "TRAMPOLINE_BUILD_FILE"
10+
value: "github/python-pubsub/.kokoro/test-samples-against-head.sh"
11+
}

‎packages/google-cloud-memcache/.kokoro/samples/python3.7/periodic-head.cfg‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Format: //devtools/kokoro/config/proto/build.proto
2+
3+
env_vars: {
4+
key: "INSTALL_LIBRARY_FROM_SOURCE"
5+
value: "True"
6+
}
7+
8+
env_vars: {
9+
key: "TRAMPOLINE_BUILD_FILE"
10+
value: "github/python-pubsub/.kokoro/test-samples-against-head.sh"
11+
}

‎packages/google-cloud-memcache/.kokoro/samples/python3.8/periodic-head.cfg‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Format: //devtools/kokoro/config/proto/build.proto
2+
3+
env_vars: {
4+
key: "INSTALL_LIBRARY_FROM_SOURCE"
5+
value: "True"
6+
}
7+
8+
env_vars: {
9+
key: "TRAMPOLINE_BUILD_FILE"
10+
value: "github/python-pubsub/.kokoro/test-samples-against-head.sh"
11+
}

‎packages/google-cloud-memcache/.kokoro/test-samples-against-head.sh‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/bin/bash
2+
# Copyright 2020 Google LLC
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# https://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
# A customized test runner for samples.
17+
#
18+
# For periodic builds, you can specify this file for testing against head.
19+
20+
# `-e` enables the script to automatically fail when a command fails
21+
# `-o pipefail` sets the exit code to the rightmost comment to exit with a non-zero
22+
set -eo pipefail
23+
# Enables `**` to include files nested inside sub-folders
24+
shopt -s globstar
25+
26+
cd github/python-memcache
27+
28+
exec .kokoro/test-samples-impl.sh

‎packages/google-cloud-memcache/.kokoro/test-samples-impl.sh‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
#!/bin/bash
2+
# Copyright 2021 Google LLC
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# https://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
17+
# `-e` enables the script to automatically fail when a command fails
18+
# `-o pipefail` sets the exit code to the rightmost comment to exit with a non-zero
19+
set -eo pipefail
20+
# Enables `**` to include files nested inside sub-folders
21+
shopt -s globstar
22+
23+
# Exit early if samples directory doesn't exist
24+
if [ ! -d "./samples" ]; then
25+
echo "No tests run. `./samples` not found"
26+
exit 0
27+
fi
28+
29+
# Disable buffering, so that the logs stream through.
30+
export PYTHONUNBUFFERED=1
31+
32+
# Debug: show build environment
33+
env | grep KOKORO
34+
35+
# Install nox
36+
python3.6 -m pip install --upgrade --quiet nox
37+
38+
# Use secrets acessor service account to get secrets
39+
if [[ -f "${KOKORO_GFILE_DIR}/secrets_viewer_service_account.json" ]]; then
40+
gcloud auth activate-service-account \
41+
--key-file="${KOKORO_GFILE_DIR}/secrets_viewer_service_account.json" \
42+
--project="cloud-devrel-kokoro-resources"
43+
fi
44+
45+
# This script will create 3 files:
46+
# - testing/test-env.sh
47+
# - testing/service-account.json
48+
# - testing/client-secrets.json
49+
./scripts/decrypt-secrets.sh
50+
51+
source ./testing/test-env.sh
52+
export GOOGLE_APPLICATION_CREDENTIALS=$(pwd)/testing/service-account.json
53+
54+
# For cloud-run session, we activate the service account for gcloud sdk.
55+
gcloud auth activate-service-account \
56+
--key-file "${GOOGLE_APPLICATION_CREDENTIALS}"
57+
58+
export GOOGLE_CLIENT_SECRETS=$(pwd)/testing/client-secrets.json
59+
60+
echo -e "\n******************** TESTING PROJECTS ********************"
61+
62+
# Switch to 'fail at end' to allow all tests to complete before exiting.
63+
set +e
64+
# Use RTN to return a non-zero value if the test fails.
65+
RTN=0
66+
ROOT=$(pwd)
67+
# Find all requirements.txt in the samples directory (may break on whitespace).
68+
for file in samples/**/requirements.txt; do
69+
cd "$ROOT"
70+
# Navigate to the project folder.
71+
file=$(dirname "$file")
72+
cd "$file"
73+
74+
echo "------------------------------------------------------------"
75+
echo "- testing $file"
76+
echo "------------------------------------------------------------"
77+
78+
# Use nox to execute the tests for the project.
79+
python3.6 -m nox -s "$RUN_TESTS_SESSION"
80+
EXIT=$?
81+
82+
# If this is a periodic build, send the test log to the FlakyBot.
83+
# See https://github.com/googleapis/repo-automation-bots/tree/master/packages/flakybot.
84+
if [[ $KOKORO_BUILD_ARTIFACTS_SUBDIR = *"periodic"* ]]; then
85+
chmod +x $KOKORO_GFILE_DIR/linux_amd64/flakybot
86+
$KOKORO_GFILE_DIR/linux_amd64/flakybot
87+
fi
88+
89+
if [[ $EXIT -ne 0 ]]; then
90+
RTN=1
91+
echo -e "\n Testing failed: Nox returned a non-zero exit code. \n"
92+
else
93+
echo -e "\n Testing completed.\n"
94+
fi
95+
96+
done
97+
cd "$ROOT"
98+
99+
# Workaround for Kokoro permissions issue: delete secrets
100+
rm testing/{test-env.sh,client-secrets.json,service-account.json}
101+
102+
exit "$RTN"

‎packages/google-cloud-memcache/.kokoro/test-samples.sh‎

Lines changed: 16 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@
1313
# See the License for the specific language governing permissions and
1414
# limitations under the License.
1515

16+
# The default test runner for samples.
17+
#
18+
# For periodic builds, we rewinds the repo to the latest release, and
19+
# run test-samples-impl.sh.
1620

1721
# `-e` enables the script to automatically fail when a command fails
1822
# `-o pipefail` sets the exit code to the rightmost comment to exit with a non-zero
@@ -24,87 +28,19 @@ cd github/python-memcache
2428

2529
# Run periodic samples tests at latest release
2630
if [[ $KOKORO_BUILD_ARTIFACTS_SUBDIR = *"periodic"* ]]; then
31+
# preserving the test runner implementation.
32+
cp .kokoro/test-samples-impl.sh "${TMPDIR}/test-samples-impl.sh"
33+
echo "--- IMPORTANT IMPORTANT IMPORTANT ---"
34+
echo "Now we rewind the repo back to the latest release..."
2735
LATEST_RELEASE=$(git describe --abbrev=0 --tags)
2836
git checkout $LATEST_RELEASE
29-
fi
30-
31-
# Exit early if samples directory doesn't exist
32-
if [ ! -d "./samples" ]; then
33-
echo "No tests run. `./samples` not found"
34-
exit 0
35-
fi
36-
37-
# Disable buffering, so that the logs stream through.
38-
export PYTHONUNBUFFERED=1
39-
40-
# Debug: show build environment
41-
env | grep KOKORO
42-
43-
# Install nox
44-
python3.6 -m pip install --upgrade --quiet nox
45-
46-
# Use secrets acessor service account to get secrets
47-
if [[ -f "${KOKORO_GFILE_DIR}/secrets_viewer_service_account.json" ]]; then
48-
gcloud auth activate-service-account \
49-
--key-file="${KOKORO_GFILE_DIR}/secrets_viewer_service_account.json" \
50-
--project="cloud-devrel-kokoro-resources"
51-
fi
52-
53-
# This script will create 3 files:
54-
# - testing/test-env.sh
55-
# - testing/service-account.json
56-
# - testing/client-secrets.json
57-
./scripts/decrypt-secrets.sh
58-
59-
source ./testing/test-env.sh
60-
export GOOGLE_APPLICATION_CREDENTIALS=$(pwd)/testing/service-account.json
61-
62-
# For cloud-run session, we activate the service account for gcloud sdk.
63-
gcloud auth activate-service-account \
64-
--key-file "${GOOGLE_APPLICATION_CREDENTIALS}"
65-
66-
export GOOGLE_CLIENT_SECRETS=$(pwd)/testing/client-secrets.json
67-
68-
echo -e "\n******************** TESTING PROJECTS ********************"
69-
70-
# Switch to 'fail at end' to allow all tests to complete before exiting.
71-
set +e
72-
# Use RTN to return a non-zero value if the test fails.
73-
RTN=0
74-
ROOT=$(pwd)
75-
# Find all requirements.txt in the samples directory (may break on whitespace).
76-
for file in samples/**/requirements.txt; do
77-
cd "$ROOT"
78-
# Navigate to the project folder.
79-
file=$(dirname "$file")
80-
cd "$file"
81-
82-
echo "------------------------------------------------------------"
83-
echo "- testing $file"
84-
echo "------------------------------------------------------------"
85-
86-
# Use nox to execute the tests for the project.
87-
python3.6 -m nox -s "$RUN_TESTS_SESSION"
88-
EXIT=$?
89-
90-
# If this is a periodic build, send the test log to the FlakyBot.
91-
# See https://github.com/googleapis/repo-automation-bots/tree/master/packages/flakybot.
92-
if [[ $KOKORO_BUILD_ARTIFACTS_SUBDIR = *"periodic"* ]]; then
93-
chmod +x $KOKORO_GFILE_DIR/linux_amd64/flakybot
94-
$KOKORO_GFILE_DIR/linux_amd64/flakybot
37+
echo "The current head is: "
38+
echo $(git rev-parse --verify HEAD)
39+
echo "--- IMPORTANT IMPORTANT IMPORTANT ---"
40+
# move back the test runner implementation if there's no file.
41+
if [ ! -f .kokoro/test-samples-impl.sh ]; then
42+
cp "${TMPDIR}/test-samples-impl.sh" .kokoro/test-samples-impl.sh
9543
fi
44+
fi
9645

97-
if [[ $EXIT -ne 0 ]]; then
98-
RTN=1
99-
echo -e "\n Testing failed: Nox returned a non-zero exit code. \n"
100-
else
101-
echo -e "\n Testing completed.\n"
102-
fi
103-
104-
done
105-
cd "$ROOT"
106-
107-
# Workaround for Kokoro permissions issue: delete secrets
108-
rm testing/{test-env.sh,client-secrets.json,service-account.json}
109-
110-
exit "$RTN"
46+
exec .kokoro/test-samples-impl.sh

‎packages/google-cloud-memcache/.pre-commit-config.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,6 @@ repos:
1212
hooks:
1313
- id: black
1414
- repo: https://gitlab.com/pycqa/flake8
15-
rev: 3.8.4
15+
rev: 3.9.0
1616
hooks:
1717
- id: flake8

‎packages/google-cloud-memcache/LICENSE‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
1-
Apache License
1+
2+
Apache License
23
Version 2.0, January 2004
3-
https://www.apache.org/licenses/
4+
http://www.apache.org/licenses/
45

56
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
67

@@ -192,7 +193,7 @@
192193
you may not use this file except in compliance with the License.
193194
You may obtain a copy of the License at
194195

195-
https://www.apache.org/licenses/LICENSE-2.0
196+
http://www.apache.org/licenses/LICENSE-2.0
196197

197198
Unless required by applicable law or agreed to in writing, software
198199
distributed under the License is distributed on an "AS IS" BASIS,

0 commit comments

Comments
 (0)