Skip to content

Commit eb19469

Browse files
odeke-emodeke-em
committed
.kokoro: add parallel builds/workers
Phase 4 of getting CI to work with Kokoro: * Add django test apps into a single file, sorted alphabetically * Adjust the per machine parallelizer to accept a job index ID and count and then grab the chunk of tests for it * Parallelizes tests on multiple cores using multiple workers * Spanner's admin limits are 5QPs over 100 seconds for administrative client usage: meaning 1 request every 20 seconds. This change attempts to introduce jitter into how fast tests begin with the hope of preventing those limits. * Limits parallelization to 4 jobs to reduce flakes from quota exhaustion errors * django_spanner: skip file_uploads.*.test_readonly_root, per #407 * django_spanner: skip migration_test_data.*.test_persistence, per #408 Reported issue #407. Reported issue #408. Updates #289.
1 parent 9d977be commit eb19469

27 files changed

+268
-104
lines changed

‎packages/django-google-spanner/.kokoro/build.sh‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -37,16 +37,21 @@ python3.6 -m tox
3737
python3.6 -m isort --recursive --check-only --diff
3838
python3.6 -m flake8
3939

40-
# Run with the Django test apps.
40+
# Export essential environment variables for Django tests.
4141
export RUNNING_SPANNER_BACKEND_TESTS=1
42-
export DJANGO_TEST_APPS="admin_changelist admin_custom_urls admin_docs admin_inlines admin_ordering aggregation aggregation_regress annotations backends basic bulk_create cache choices custom_columns indexes inline_formsets introspection invalid_models_tests known_related_objects lookup max_lengths m2m_and_m2o m2m_intermediary m2m_multiple m2m_recursive m2m_regress m2m_signals m2m_through m2m_through_regress m2o_recursive managers_regress many_to_many many_to_one many_to_one_null max_lengths migrate_signals migrations.test_operations migration_test_data_persistence"
42+
# Hardcode the max number of workers since Spanner has issues
43+
# with a very low QPS for administrative RPCs of 5QPS (averaged every 100 seconds)
44+
export DJANGO_WORKER_COUNT=4 # $(ls .kokoro/presubmit/worker* | wc -l)
4345

4446
pip3 install .
45-
mkdir -p django_tests && git clone --depth 1 --single-branch --branch spanner-2.2.x https://github.com/timgraham/django.git django_tests/django
46-
# cd django_tests/django && pip3 install -e .; cd ../../
47+
# Create a unique DJANGO_TESTS_DIR per worker to avoid
48+
# any clashes with configured tests by other workers.
49+
export DJANGO_TESTS_DIR="django_tests_$DJANGO_WORKER_INDEX"
50+
mkdir -p $DJANGO_TESTS_DIR && git clone --depth 1 --single-branch --branch spanner-2.2.x https://github.com/timgraham/django.git $DJANGO_TESTS_DIR/django
4751

4852
# Install dependencies for Django tests.
4953
sudo apt-get update
5054
apt-get install -y libffi-dev libjpeg-dev zlib1g-dev libmemcached-dev
51-
cd django_tests/django && pip3 install -e . && pip3 install -r tests/requirements/py3.txt; cd ../../
55+
cd $DJANGO_TESTS_DIR/django && pip3 install -e . && pip3 install -r tests/requirements/py3.txt; cd ../../
56+
5257
./bin/parallelize_tests_linux

‎packages/django-google-spanner/.kokoro/continuous/admin_to_annotations.cfg‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎packages/django-google-spanner/.kokoro/continuous/backends_to_cache.cfg‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎packages/django-google-spanner/.kokoro/continuous/worker_0.cfg‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
env_vars: {
2+
key: "DJANGO_WORKER_INDEX"
3+
value: "0"
4+
}

‎packages/django-google-spanner/.kokoro/continuous/worker_1.cfg‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
env_vars: {
2+
key: "DJANGO_WORKER_INDEX"
3+
value: "1"
4+
}

‎packages/django-google-spanner/.kokoro/continuous/worker_2.cfg‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
env_vars: {
2+
key: "DJANGO_WORKER_INDEX"
3+
value: "2"
4+
}

‎packages/django-google-spanner/.kokoro/continuous/worker_3.cfg‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
env_vars: {
2+
key: "DJANGO_WORKER_INDEX"
3+
value: "3"
4+
}

‎packages/django-google-spanner/.kokoro/continuous/worker_4.cfg‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
env_vars: {
2+
key: "DJANGO_WORKER_INDEX"
3+
value: "4"
4+
}

‎packages/django-google-spanner/.kokoro/continuous/worker_5.cfg‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
env_vars: {
2+
key: "DJANGO_WORKER_INDEX"
3+
value: "5"
4+
}

‎packages/django-google-spanner/.kokoro/continuous/worker_6.cfg‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
env_vars: {
2+
key: "DJANGO_WORKER_INDEX"
3+
value: "6"
4+
}

0 commit comments

Comments
 (0)