Skip to content

Commit 17a1573

Browse files
f0rmigaf0rmiga
authored
feat: multi-toolchain support (bazel-contrib#846)
* feat: multi-toolchain support This adds support for multiple Python versions on the same Bazel workspace. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: cross-version testing A py_test using 3.10 runs a py_binary using 3.9. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: error message Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * doc: add link to bazelbuild/bazel PR fixing expand_location Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: set environment variables for py_binary too Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * test: extra case for default version taking another version Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: fail if args attribute is set Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: remove confusing output with same target name Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: buildifier Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * revert: use testing.TestEnvironment See comment in code for the reasons. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: linting issues Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: black linter Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: move tests to a sub-dir Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: add multi_pip_parse Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: add missing aliases Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: use requirement function in example Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: deleted packages Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: update generated docs Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: version checking of the rule is already done by other tests Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: add python_interpreter_target to multi_pip_parse Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: windows Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: unify py_test and py_binary transition impls Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: test compatible with all platforms Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * rebase: adjust multi_python_versions on ci Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: use usr flags instead of platforms in transition Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: rename rule -> rule_impl This avoids confusion with the global `rule` https://bazel.build/rules/lib/globals#rule. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: reduce repetition of args Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: missing test and binary-specific attributes Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: add srcs and deps attrs for path expansion Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: missing bazel_skylib on integration tests Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: use ctx.target_platform_has_constraint over select Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * doc: why symlink <name>.zip under Windows Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: apply suggestions from code review Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: use incoming edge transitions Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: use RunEnvironmentInfo when available Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: cfg should be target not exec Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>
1 parent 0054574 commit 17a1573

31 files changed

Lines changed: 1251 additions & 51 deletions

‎.bazelci/presubmit.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,22 @@ tasks:
8585
working_directory: examples/bzlmod
8686
platform: windows
8787

88+
integration_test_multi_python_versions_linux:
89+
<<: *reusable_build_test_all
90+
name: multi_python_versions integration tests on Linux
91+
working_directory: examples/multi_python_versions
92+
platform: ubuntu2004
93+
integration_test_multi_python_versions_macos:
94+
<<: *reusable_build_test_all
95+
name: multi_python_versions integration tests on macOS
96+
working_directory: examples/multi_python_versions
97+
platform: macos
98+
integration_test_multi_python_versions_windows:
99+
<<: *reusable_build_test_all
100+
name: multi_python_versions integration tests on Windows
101+
working_directory: examples/multi_python_versions
102+
platform: windows
103+
88104
integration_test_pip_install_linux:
89105
<<: *reusable_build_test_all
90106
name: pip_install integration tests on Linux

‎.bazelrc‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
# This lets us glob() up all the files inside the examples to make them inputs to tests
44
# (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it)
55
# To update these lines, run tools/bazel_integration_test/update_deleted_packages.sh
6-
build --deleted_packages=examples/build_file_generation,examples/bzlmod,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_import,examples/relative_requirements,tests/pip_repository_entry_points,tests/pip_deps
7-
query --deleted_packages=examples/build_file_generation,examples/bzlmod,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_import,examples/relative_requirements,tests/pip_repository_entry_points,tests/pip_deps
6+
build --deleted_packages=examples/build_file_generation,examples/bzlmod,examples/multi_python_versions,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_import,examples/relative_requirements,tests/pip_repository_entry_points,tests/pip_deps
7+
query --deleted_packages=examples/build_file_generation,examples/bzlmod,examples/multi_python_versions,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_import,examples/relative_requirements,tests/pip_repository_entry_points,tests/pip_deps
88

99
test --test_output=errors
1010

‎WORKSPACE‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,13 +25,13 @@ load("//:internal_setup.bzl", "rules_python_internal_setup")
2525

2626
rules_python_internal_setup()
2727

28-
load("//python:repositories.bzl", "python_register_toolchains")
28+
load("//python:repositories.bzl", "python_register_multi_toolchains")
2929
load("//python:versions.bzl", "MINOR_MAPPING")
3030

31-
python_register_toolchains(
31+
python_register_multi_toolchains(
3232
name = "python",
33-
# We always use the latest Python internally.
34-
python_version = MINOR_MAPPING.values()[-1],
33+
default_version = MINOR_MAPPING.values()[-1],
34+
python_versions = MINOR_MAPPING.values(),
3535
)
3636

3737
load("//gazelle:deps.bzl", "gazelle_deps")

‎docs/pip.md‎

Lines changed: 67 additions & 10 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/BUILD‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,11 @@ bazel_integration_test(
3838
timeout = "long",
3939
)
4040

41+
bazel_integration_test(
42+
name = "multi_python_versions_example",
43+
timeout = "long",
44+
)
45+
4146
bazel_integration_test(
4247
name = "bzlmod_example",
4348
bzlmod = True,

‎examples/multi_python_versions/.bazelrc‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
test --test_output=errors
2+
3+
# Windows requires these for multi-python support:
4+
build --enable_runfiles
5+
startup --windows_enable_symlinks

‎examples/multi_python_versions/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
bazel-*

‎examples/multi_python_versions/WORKSPACE‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
workspace(name = "rules_python_multi_python_versions")
2+
3+
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
4+
5+
http_archive(
6+
name = "bazel_skylib",
7+
sha256 = "c6966ec828da198c5d9adbaa94c05e3a1c7f21bd012a0b29ba8ddbccb2c93b0d",
8+
urls = [
9+
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz",
10+
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz",
11+
],
12+
)
13+
14+
local_repository(
15+
name = "rules_python",
16+
path = "../..",
17+
)
18+
19+
load("@rules_python//python/pip_install:repositories.bzl", "pip_install_dependencies")
20+
21+
pip_install_dependencies()
22+
23+
load("@rules_python//python:repositories.bzl", "python_register_multi_toolchains")
24+
25+
default_python_version = "3.9"
26+
27+
python_register_multi_toolchains(
28+
name = "python",
29+
default_version = default_python_version,
30+
python_versions = [
31+
"3.8",
32+
"3.9",
33+
"3.10",
34+
],
35+
)
36+
37+
load("@python//:pip.bzl", "multi_pip_parse")
38+
load("@python//3.10:defs.bzl", interpreter_3_10 = "interpreter")
39+
load("@python//3.8:defs.bzl", interpreter_3_8 = "interpreter")
40+
load("@python//3.9:defs.bzl", interpreter_3_9 = "interpreter")
41+
42+
multi_pip_parse(
43+
name = "pypi",
44+
default_version = default_python_version,
45+
python_interpreter_target = {
46+
"3.10": interpreter_3_10,
47+
"3.8": interpreter_3_8,
48+
"3.9": interpreter_3_9,
49+
},
50+
requirements_lock = {
51+
"3.10": "//requirements:requirements_lock_3_10.txt",
52+
"3.8": "//requirements:requirements_lock_3_8.txt",
53+
"3.9": "//requirements:requirements_lock_3_9.txt",
54+
},
55+
)
56+
57+
load("@pypi//:requirements.bzl", "install_deps")
58+
59+
install_deps()

‎examples/multi_python_versions/libs/my_lib/BUILD.bazel‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
load("@pypi//:requirements.bzl", "requirement")
2+
load("@rules_python//python:defs.bzl", "py_library")
3+
4+
py_library(
5+
name = "my_lib",
6+
srcs = ["__init__.py"],
7+
visibility = ["@//tests:__pkg__"],
8+
deps = [requirement("websockets")],
9+
)

‎examples/multi_python_versions/libs/my_lib/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
import websockets
2+
3+
4+
def websockets_is_for_python_version(sanitized_version_check):
5+
return f"pypi_{sanitized_version_check}_websockets" in websockets.__file__

0 commit comments

Comments
 (0)